2023-11-04T05:15:35.221 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-04T05:15:35.222 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --name rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447118 --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447118/orig.config.yaml 2023-11-04T05:15:35.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T05:15:35.359 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447118 branch: rishabh-2023nov3 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: null first_in_suite: false job_id: '7447118' kernel: client: sha1: distro flavor: default kdb: true sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-2023nov3 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 826ae4205210243850fc3f7bc3f8aea0d1f5ced2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 826ae4205210243850fc3f7bc3f8aea0d1f5ced2 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023nov3 sha1: 826ae4205210243850fc3f7bc3f8aea0d1f5ced2 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9821 sha1: 826ae4205210243850fc3f7bc3f8aea0d1f5ced2 sleep_before_teardown: 0 subset: 332/512 suite: fs suite_branch: rishabh-2023nov3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_826ae4205210243850fc3f7bc3f8aea0d1f5ced2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 826ae4205210243850fc3f7bc3f8aea0d1f5ced2 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-04_04:30:51 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822 2023-11-04T05:15:35.360 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_826ae4205210243850fc3f7bc3f8aea0d1f5ced2/qa; will attempt to use it 2023-11-04T05:15:35.361 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_826ae4205210243850fc3f7bc3f8aea0d1f5ced2/qa/tasks 2023-11-04T05:15:35.361 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-04T05:15:35.363 INFO:teuthology.task.internal:Checking packages... 2023-11-04T05:15:35.387 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '826ae4205210243850fc3f7bc3f8aea0d1f5ced2' 2023-11-04T05:15:35.388 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-04T05:15:35.388 INFO:teuthology.packaging:ref: None 2023-11-04T05:15:35.389 INFO:teuthology.packaging:tag: None 2023-11-04T05:15:35.389 INFO:teuthology.packaging:branch: rishabh-2023nov3 2023-11-04T05:15:35.389 INFO:teuthology.packaging:sha1: 826ae4205210243850fc3f7bc3f8aea0d1f5ced2 2023-11-04T05:15:35.390 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-2023nov3 2023-11-04T05:15:35.607 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7050.g826ae420 2023-11-04T05:15:35.607 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-04T05:15:35.614 INFO:teuthology.task.internal:no buildpackages task found 2023-11-04T05:15:35.615 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-04T05:15:35.620 INFO:teuthology.task.internal:Saving configuration 2023-11-04T05:15:35.637 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-04T05:15:35.643 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-04T05:15:35.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447118', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 05:10:38.339974', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T05:15:35.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447118', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 05:10:38.338161', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T05:15:35.688 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-04T05:15:35.694 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-04T05:15:35.694 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-04T05:15:35.694 INFO:teuthology.run_tasks:Running task console_log... 2023-11-04T05:15:35.764 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f23ef4c7b80>, signals=[15]) 2023-11-04T05:15:35.765 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-04T05:15:35.772 INFO:teuthology.task.internal:Opening connections... 2023-11-04T05:15:35.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:15:35.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T05:15:35.843 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:15:35.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T05:15:35.911 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-04T05:15:35.917 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-11-04T05:15:35.937 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-11-04T05:15:35.938 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-11-04T05:15:35.996 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T05:15:35.996 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T05:15:35.996 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel" 2023-11-04T05:15:35.997 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora" 2023-11-04T05:15:35.997 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.6" 2023-11-04T05:15:35.997 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8" 2023-11-04T05:15:35.997 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T05:15:35.997 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2023-11-04T05:15:35.998 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T05:15:35.998 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T05:15:35.998 INFO:teuthology.orchestra.run.smithi003.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T05:15:35.998 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T05:15:35.998 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:15:35.999 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T05:15:35.999 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T05:15:35.999 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T05:15:35.999 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T05:15:36.000 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-11-04T05:15:36.029 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2023-11-04T05:15:36.049 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2023-11-04T05:15:36.050 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-11-04T05:15:36.109 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T05:15:36.109 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T05:15:36.110 INFO:teuthology.orchestra.run.smithi188.stdout:ID="rhel" 2023-11-04T05:15:36.110 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="fedora" 2023-11-04T05:15:36.110 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8.6" 2023-11-04T05:15:36.110 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2023-11-04T05:15:36.111 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T05:15:36.111 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-11-04T05:15:36.111 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T05:15:36.111 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T05:15:36.111 INFO:teuthology.orchestra.run.smithi188.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T05:15:36.112 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T05:15:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:15:36.112 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T05:15:36.112 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T05:15:36.112 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T05:15:36.113 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T05:15:36.113 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-11-04T05:15:36.137 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-04T05:15:36.149 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-04T05:15:36.271 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-04T05:15:36.271 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2023-11-04T05:15:36.274 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2023-11-04T05:15:36.293 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-04T05:15:36.300 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-04T05:15:36.300 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2023-11-04T05:15:36.334 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2023-11-04T05:15:36.362 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-04T05:15:36.420 INFO:teuthology.run_tasks:Running task kernel... 2023-11-04T05:15:36.434 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'} 2023-11-04T05:15:36.435 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-04T05:15:36.436 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-04T05:15:36.436 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-04T05:15:36.437 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-04T05:15:36.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:15:36.456 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-11-04T05:15:36.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:15:36.457 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-11-04T05:15:36.514 INFO:teuthology.orchestra.run.smithi188.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:36.515 INFO:teuthology.task.kernel:Running kernel on smithi188: 4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:36.515 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2023-11-04T05:15:36.518 INFO:teuthology.orchestra.run.smithi003.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:36.518 INFO:teuthology.task.kernel:Running kernel on smithi003: 4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:36.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2023-11-04T05:15:37.714 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-11-04T05:15:37.715 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-11-04T05:15:37.737 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2023-11-04T05:15:37.738 INFO:teuthology.orchestra.run.smithi188.stdout:Unable to read consumer identity 2023-11-04T05:15:39.117 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-11-04T05:15:39.129 INFO:teuthology.orchestra.run.smithi188.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-11-04T05:15:39.871 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-04T05:15:39.976 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-11-04T05:15:43.380 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 410 kB/s | 24 kB 00:00 2023-11-04T05:15:43.660 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 410 kB/s | 24 kB 00:00 2023-11-04T05:15:44.083 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-04T05:15:44.147 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-11-04T05:15:44.148 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2023-11-04T05:15:44.148 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-11-04T05:15:44.266 DEBUG:teuthology.orchestra.run.smithi188:> echo no | sudo yum reinstall kernel || true 2023-11-04T05:15:44.412 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-04T05:15:44.462 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-11-04T05:15:44.464 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2023-11-04T05:15:44.464 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-11-04T05:15:44.577 DEBUG:teuthology.orchestra.run.smithi003:> echo no | sudo yum reinstall kernel || true 2023-11-04T05:15:44.597 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2023-11-04T05:15:44.598 INFO:teuthology.orchestra.run.smithi188.stdout:Unable to read consumer identity 2023-11-04T05:15:44.865 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:01 ago on Sat 04 Nov 2023 05:15:43 AM UTC. 2023-11-04T05:15:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-04T05:15:44.926 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-11-04T05:15:44.926 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-11-04T05:15:44.943 INFO:teuthology.orchestra.run.smithi188.stderr:Error: No packages marked for reinstall. 2023-11-04T05:15:44.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum reinstall -y kernel || true 2023-11-04T05:15:45.206 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 Nov 2023 05:15:43 AM UTC. 2023-11-04T05:15:45.252 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-04T05:15:45.288 INFO:teuthology.orchestra.run.smithi003.stderr:Error: No packages marked for reinstall. 2023-11-04T05:15:45.320 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2023-11-04T05:15:45.320 INFO:teuthology.orchestra.run.smithi188.stdout:Unable to read consumer identity 2023-11-04T05:15:45.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum reinstall -y kernel || true 2023-11-04T05:15:45.582 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 Nov 2023 05:15:43 AM UTC. 2023-11-04T05:15:45.628 INFO:teuthology.orchestra.run.smithi188.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-04T05:15:45.654 INFO:teuthology.orchestra.run.smithi188.stderr:Error: No packages marked for reinstall. 2023-11-04T05:15:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-11-04T05:15:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-11-04T05:15:45.705 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2023-11-04T05:15:45.786 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:45.787 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:45.787 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-04T05:15:45.788 DEBUG:teuthology.parallel:result is None 2023-11-04T05:15:45.954 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 Nov 2023 05:15:43 AM UTC. 2023-11-04T05:15:45.995 INFO:teuthology.orchestra.run.smithi003.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-04T05:15:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:Error: No packages marked for reinstall. 2023-11-04T05:15:46.069 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2023-11-04T05:15:46.152 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:46.153 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-04T05:15:46.153 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-04T05:15:46.153 DEBUG:teuthology.parallel:result is None 2023-11-04T05:15:46.153 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-04T05:15:46.161 INFO:teuthology.task.internal:Creating test directory... 2023-11-04T05:15:46.161 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-04T05:15:46.163 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-04T05:15:46.180 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-04T05:15:46.187 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-04T05:15:46.193 INFO:teuthology.task.internal:Creating archive directory... 2023-11-04T05:15:46.194 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-04T05:15:46.220 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-04T05:15:46.251 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-04T05:15:46.257 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-04T05:15:46.258 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-04T05:15:46.278 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-04T05:15:46.320 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-04T05:15:46.322 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-04T05:15:46.332 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-04T05:15:46.342 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-04T05:15:46.343 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-04T05:15:46.350 INFO:teuthology.task.internal:Configuring sudo... 2023-11-04T05:15:46.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-04T05:15:46.353 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-04T05:15:46.400 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-04T05:15:46.409 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-04T05:15:46.409 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-04T05:15:46.421 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-04T05:15:46.457 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-04T05:15:46.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-04T05:15:46.568 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-04T05:15:46.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-04T05:15:46.693 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:15:46.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-04T05:15:46.761 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-04T05:15:46.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-04T05:15:46.854 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-04T05:15:46.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-04T05:15:46.980 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:15:46.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-04T05:15:47.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2023-11-04T05:15:47.050 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2023-11-04T05:15:47.107 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-04T05:15:47.165 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-04T05:15:47.545 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-04T05:15:47.553 INFO:teuthology.task.internal:Starting timer... 2023-11-04T05:15:47.553 INFO:teuthology.run_tasks:Running task pcp... 2023-11-04T05:15:47.563 INFO:teuthology.run_tasks:Running task selinux... 2023-11-04T05:15:47.572 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-04T05:15:47.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2023-11-04T05:15:47.642 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2023-11-04T05:15:47.643 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2023-11-04T05:15:47.692 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2023-11-04T05:15:47.694 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-04T05:15:47.695 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2023-11-04T05:15:47.718 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2023-11-04T05:15:47.718 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2023-11-04T05:15:47.739 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2023-11-04T05:15:47.740 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2023-11-04T05:15:47.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-04T05:15:47.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:15:47.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-04T05:15:47.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:15:47.813 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-04T05:15:47.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2023-11-04T05:15:47.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2023-11-04T05:15:47.892 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-04T05:15:47.902 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-04T05:15:47.907 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-04T05:15:48.016 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-04T05:15:48.033 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-04T05:15:48.035 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-04T05:24:59.161 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2023-11-04T05:24:59.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-11-04T05:24:59.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T05:24:59.235 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-11-04T05:24:59.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-11-04T05:24:59.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-04T05:24:59.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T05:24:59.375 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-11-04T05:24:59.445 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-04T05:24:59.446 INFO:teuthology.run_tasks:Running task clock... 2023-11-04T05:24:59.460 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-04T05:24:59.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-04T05:24:59.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-04T05:24:59.464 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-04T05:24:59.464 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-04T05:24:59.498 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-04T05:24:59.515 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-04T05:24:59.519 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-04T05:24:59.534 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-04T05:24:59.584 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2023-11-04T05:24:59.597 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2023-11-04T05:24:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-04T05:24:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2023-11-04T05:24:59.627 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-04T05:24:59.636 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2023-11-04T05:24:59.650 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-04T05:24:59.664 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-04T05:24:59.699 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-11-04T05:24:59.702 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-04T05:24:59.703 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-11-04T05:24:59.703 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.703 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.703 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.704 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:bash: ntpq: command not found 2023-11-04T05:24:59.738 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-04T05:24:59.738 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2023-11-04T05:24:59.738 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.739 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.739 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.739 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-04T05:24:59.740 INFO:teuthology.run_tasks:Running task install... 2023-11-04T05:24:59.751 DEBUG:teuthology.task.install:project ceph 2023-11-04T05:24:59.751 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2'}} 2023-11-04T05:24:59.752 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2'} 2023-11-04T05:24:59.752 INFO:teuthology.task.install:Using flavor: default 2023-11-04T05:24:59.761 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-04T05:24:59.762 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-04T05:24:59.766 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2', 'tag': None, 'wait_for_package': False} 2023-11-04T05:24:59.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=826ae4205210243850fc3f7bc3f8aea0d1f5ced2 2023-11-04T05:24:59.770 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2', 'tag': None, 'wait_for_package': False} 2023-11-04T05:24:59.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=826ae4205210243850fc3f7bc3f8aea0d1f5ced2 2023-11-04T05:24:59.912 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/ 2023-11-04T05:24:59.913 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7050.g826ae420 2023-11-04T05:24:59.985 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/ 2023-11-04T05:24:59.985 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7050.g826ae420 2023-11-04T05:25:00.086 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-04T05:25:00.087 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:25:00.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-04T05:25:00.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023nov3/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-04T05:25:00.098 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:25:00.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-04T05:25:00.118 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-04T05:25:00.118 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-04T05:25:00.126 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-04T05:25:00.127 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/826ae4205210243850fc3f7bc3f8aea0d1f5ced2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-04T05:25:00.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-04T05:25:00.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-04T05:25:00.270 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-04T05:25:00.273 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-04T05:25:00.338 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2023-11-04T05:25:00.339 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2023-11-04T05:25:00.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2023-11-04T05:25:00.341 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2023-11-04T05:25:00.714 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2023-11-04T05:25:00.736 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-11-04T05:25:00.939 INFO:teuthology.orchestra.run.smithi188.stdout:44 files removed 2023-11-04T05:25:00.983 INFO:teuthology.orchestra.run.smithi003.stdout:44 files removed 2023-11-04T05:25:00.986 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-04T05:25:01.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-04T05:25:01.316 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2023-11-04T05:25:01.383 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-11-04T05:25:02.146 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 202 kB/s | 81 kB 00:00 2023-11-04T05:25:02.173 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 160 kB/s | 81 kB 00:00 2023-11-04T05:25:02.397 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 44 kB/s | 9.4 kB 00:00 2023-11-04T05:25:02.549 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-11-04T05:25:02.740 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2023-11-04T05:25:02.873 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-11-04T05:25:02.916 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2023-11-04T05:25:03.059 INFO:teuthology.orchestra.run.smithi188.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-04T05:25:07.301 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 16 MB 00:04 2023-11-04T05:25:09.967 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 2.3 MB/s | 16 MB 00:06 2023-11-04T05:25:11.082 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 409 kB/s | 24 kB 00:00 2023-11-04T05:25:12.092 INFO:teuthology.orchestra.run.smithi003.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-11-04T05:25:13.483 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-11-04T05:25:14.351 INFO:teuthology.orchestra.run.smithi188.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 66 MB/s | 53 MB 00:00 2023-11-04T05:25:20.777 INFO:teuthology.orchestra.run.smithi003.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 51 MB/s | 47 MB 00:00 2023-11-04T05:25:22.418 INFO:teuthology.orchestra.run.smithi188.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-11-04T05:25:32.376 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-04T05:25:32.377 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-04T05:25:32.392 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-04T05:25:32.393 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-04T05:25:32.394 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-04T05:25:32.395 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-04T05:25:32.398 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-04T05:25:32.401 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-04T05:25:32.402 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-04T05:25:32.544 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-11-04T05:25:32.558 INFO:teuthology.orchestra.run.smithi003.stdout:=========================================================================================================================== 2023-11-04T05:25:32.558 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-11-04T05:25:32.558 INFO:teuthology.orchestra.run.smithi003.stdout:=========================================================================================================================== 2023-11-04T05:25:32.558 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2023-11-04T05:25:32.559 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-04T05:25:32.559 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-04T05:25:32.559 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:18.0.0-7050.g826ae420.el8 ceph 6.4 k 2023-11-04T05:25:32.560 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:18.0.0-7050.g826ae420.el8 ceph 5.4 M 2023-11-04T05:25:32.560 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:18.0.0-7050.g826ae420.el8 ceph 919 k 2023-11-04T05:25:32.560 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7050.g826ae420.el8 ceph 171 k 2023-11-04T05:25:32.561 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:18.0.0-7050.g826ae420.el8 ceph 1.6 M 2023-11-04T05:25:32.561 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 132 k 2023-11-04T05:25:32.561 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 1.8 M 2023-11-04T05:25:32.562 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 7.4 M 2023-11-04T05:25:32.562 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 48 k 2023-11-04T05:25:32.562 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:18.0.0-7050.g826ae420.el8 ceph 14 M 2023-11-04T05:25:32.562 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:18.0.0-7050.g826ae420.el8 ceph 49 M 2023-11-04T05:25:32.563 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 261 k 2023-11-04T05:25:32.563 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 191 k 2023-11-04T05:25:32.563 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:18.0.0-7050.g826ae420.el8 ceph 262 k 2023-11-04T05:25:32.563 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 18 k 2023-11-04T05:25:32.563 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-04T05:25:32.564 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-04T05:25:32.564 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-04T05:25:32.564 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:18.0.0-7050.g826ae420.el8 ceph 31 k 2023-11-04T05:25:32.565 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 839 k 2023-11-04T05:25:32.565 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:18.0.0-7050.g826ae420.el8 ceph 139 k 2023-11-04T05:25:32.566 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-04T05:25:32.566 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-04T05:25:32.566 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-04T05:25:32.566 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-04T05:25:32.571 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-04T05:25:32.571 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-04T05:25:32.571 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:18.0.0-7050.g826ae420.el8 ceph 197 k 2023-11-04T05:25:32.571 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:18.0.0-7050.g826ae420.el8 ceph 371 k 2023-11-04T05:25:32.572 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:18.0.0-7050.g826ae420.el8 ceph 363 k 2023-11-04T05:25:32.572 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:18.0.0-7050.g826ae420.el8 ceph 106 k 2023-11-04T05:25:32.572 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:18.0.0-7050.g826ae420.el8 ceph 93 k 2023-11-04T05:25:32.572 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:18.0.0-7050.g826ae420.el8 ceph 3.6 M 2023-11-04T05:25:32.572 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:18.0.0-7050.g826ae420.el8 ceph 180 k 2023-11-04T05:25:32.573 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-04T05:25:32.573 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2023-11-04T05:25:32.573 INFO:teuthology.orchestra.run.smithi003.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-04T05:25:32.573 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 3.8 M 2023-11-04T05:25:32.573 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 3.7 M 2023-11-04T05:25:32.574 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-11-04T05:25:32.574 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-04T05:25:32.574 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:18.0.0-7050.g826ae420.el8 ceph 23 M 2023-11-04T05:25:32.574 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 23 k 2023-11-04T05:25:32.574 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:18.0.0-7050.g826ae420.el8 ceph 2.3 M 2023-11-04T05:25:32.575 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 242 k 2023-11-04T05:25:32.575 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:18.0.0-7050.g826ae420.el8 ceph 4.6 M 2023-11-04T05:25:32.575 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:18.0.0-7050.g826ae420.el8 ceph 18 M 2023-11-04T05:25:32.576 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 15 k 2023-11-04T05:25:32.576 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:18.0.0-7050.g826ae420.el8 ceph 24 k 2023-11-04T05:25:32.576 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-04T05:25:32.576 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-04T05:25:32.577 INFO:teuthology.orchestra.run.smithi003.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-04T05:25:32.577 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-04T05:25:32.577 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-04T05:25:32.578 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:18.0.0-7050.g826ae420.el8 ceph 182 k 2023-11-04T05:25:32.578 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-04T05:25:32.579 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-04T05:25:32.579 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-04T05:25:32.579 INFO:teuthology.orchestra.run.smithi003.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-04T05:25:32.580 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-04T05:25:32.580 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-04T05:25:32.580 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 518 k 2023-11-04T05:25:32.580 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-04T05:25:32.581 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 6.9 M 2023-11-04T05:25:32.581 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-04T05:25:32.581 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-04T05:25:32.582 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-04T05:25:32.582 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-04T05:25:32.582 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-04T05:25:32.583 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-04T05:25:32.583 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-04T05:25:32.584 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-04T05:25:32.584 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-04T05:25:32.584 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-04T05:25:32.584 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-04T05:25:32.584 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-04T05:25:32.585 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-04T05:25:32.585 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-04T05:25:32.585 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-04T05:25:32.585 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-04T05:25:32.585 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:18.0.0-7050.g826ae420.el8 ceph 45 k 2023-11-04T05:25:32.586 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:18.0.0-7050.g826ae420.el8 ceph 125 k 2023-11-04T05:25:32.586 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-04T05:25:32.586 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-04T05:25:32.587 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-04T05:25:32.587 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-04T05:25:32.587 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-04T05:25:32.587 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-04T05:25:32.588 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-04T05:25:32.588 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-04T05:25:32.589 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-04T05:25:32.589 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-04T05:25:32.589 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-04T05:25:32.590 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-04T05:25:32.590 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-04T05:25:32.591 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-04T05:25:32.591 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-04T05:25:32.591 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-04T05:25:32.592 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-04T05:25:32.592 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-04T05:25:32.592 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-04T05:25:32.593 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-04T05:25:32.593 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-04T05:25:32.593 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-04T05:25:32.594 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-04T05:25:32.594 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-04T05:25:32.594 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-04T05:25:32.595 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-04T05:25:32.595 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-04T05:25:32.595 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-04T05:25:32.595 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-04T05:25:32.595 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-04T05:25:32.596 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-04T05:25:32.596 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-04T05:25:32.596 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-04T05:25:32.596 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-04T05:25:32.596 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-04T05:25:32.597 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-04T05:25:32.597 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-04T05:25:32.597 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-04T05:25:32.598 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-04T05:25:32.598 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-04T05:25:32.598 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-04T05:25:32.598 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-04T05:25:32.598 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-04T05:25:32.599 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-04T05:25:32.599 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-04T05:25:32.599 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-04T05:25:32.599 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-04T05:25:32.599 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-04T05:25:32.600 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-04T05:25:32.600 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling module streams: 2023-11-04T05:25:32.600 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql 10 2023-11-04T05:25:32.600 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:25:32.600 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-11-04T05:25:32.601 INFO:teuthology.orchestra.run.smithi003.stdout:=========================================================================================================================== 2023-11-04T05:25:32.601 INFO:teuthology.orchestra.run.smithi003.stdout:Install 128 Packages 2023-11-04T05:25:32.601 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 3 Packages 2023-11-04T05:25:32.601 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:25:32.602 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 209 M 2023-11-04T05:25:32.602 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-11-04T05:25:32.851 INFO:teuthology.orchestra.run.smithi003.stdout:(1/131): ceph-18.0.0-7050.g826ae420.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-11-04T05:25:33.322 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-04T05:25:33.323 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-04T05:25:33.326 INFO:teuthology.orchestra.run.smithi003.stdout:(2/131): ceph-fuse-18.0.0-7050.g826ae420.el8.x8 1.9 MB/s | 919 kB 00:00 2023-11-04T05:25:33.337 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-04T05:25:33.338 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-04T05:25:33.339 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-04T05:25:33.341 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-04T05:25:33.343 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-04T05:25:33.346 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-04T05:25:33.347 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-04T05:25:33.427 INFO:teuthology.orchestra.run.smithi003.stdout:(3/131): ceph-immutable-object-cache-18.0.0-705 1.7 MB/s | 171 kB 00:00 2023-11-04T05:25:33.479 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-11-04T05:25:33.492 INFO:teuthology.orchestra.run.smithi188.stdout:=========================================================================================================================== 2023-11-04T05:25:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2023-11-04T05:25:33.492 INFO:teuthology.orchestra.run.smithi188.stdout:=========================================================================================================================== 2023-11-04T05:25:33.493 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2023-11-04T05:25:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-04T05:25:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-04T05:25:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:18.0.0-7050.g826ae420.el8 ceph 6.4 k 2023-11-04T05:25:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:18.0.0-7050.g826ae420.el8 ceph 5.4 M 2023-11-04T05:25:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:18.0.0-7050.g826ae420.el8 ceph 919 k 2023-11-04T05:25:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7050.g826ae420.el8 ceph 171 k 2023-11-04T05:25:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:18.0.0-7050.g826ae420.el8 ceph 1.6 M 2023-11-04T05:25:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 132 k 2023-11-04T05:25:33.495 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 1.8 M 2023-11-04T05:25:33.495 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 7.4 M 2023-11-04T05:25:33.495 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 48 k 2023-11-04T05:25:33.495 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:18.0.0-7050.g826ae420.el8 ceph 14 M 2023-11-04T05:25:33.495 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:18.0.0-7050.g826ae420.el8 ceph 49 M 2023-11-04T05:25:33.496 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 261 k 2023-11-04T05:25:33.496 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 191 k 2023-11-04T05:25:33.496 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:18.0.0-7050.g826ae420.el8 ceph 262 k 2023-11-04T05:25:33.496 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 18 k 2023-11-04T05:25:33.496 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-04T05:25:33.497 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-04T05:25:33.497 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-04T05:25:33.497 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:18.0.0-7050.g826ae420.el8 ceph 31 k 2023-11-04T05:25:33.497 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 839 k 2023-11-04T05:25:33.497 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:18.0.0-7050.g826ae420.el8 ceph 139 k 2023-11-04T05:25:33.498 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-04T05:25:33.498 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-04T05:25:33.498 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-04T05:25:33.498 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-04T05:25:33.498 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-04T05:25:33.499 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-04T05:25:33.499 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:18.0.0-7050.g826ae420.el8 ceph 197 k 2023-11-04T05:25:33.499 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:18.0.0-7050.g826ae420.el8 ceph 371 k 2023-11-04T05:25:33.499 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:18.0.0-7050.g826ae420.el8 ceph 363 k 2023-11-04T05:25:33.499 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:18.0.0-7050.g826ae420.el8 ceph 106 k 2023-11-04T05:25:33.500 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:18.0.0-7050.g826ae420.el8 ceph 93 k 2023-11-04T05:25:33.500 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:18.0.0-7050.g826ae420.el8 ceph 3.6 M 2023-11-04T05:25:33.500 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:18.0.0-7050.g826ae420.el8 ceph 180 k 2023-11-04T05:25:33.500 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-04T05:25:33.500 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2023-11-04T05:25:33.501 INFO:teuthology.orchestra.run.smithi188.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-04T05:25:33.501 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 3.8 M 2023-11-04T05:25:33.501 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 3.7 M 2023-11-04T05:25:33.501 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2023-11-04T05:25:33.501 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-04T05:25:33.502 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:18.0.0-7050.g826ae420.el8 ceph 23 M 2023-11-04T05:25:33.502 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 23 k 2023-11-04T05:25:33.502 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:18.0.0-7050.g826ae420.el8 ceph 2.3 M 2023-11-04T05:25:33.502 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 242 k 2023-11-04T05:25:33.502 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:18.0.0-7050.g826ae420.el8 ceph 4.6 M 2023-11-04T05:25:33.503 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:18.0.0-7050.g826ae420.el8 ceph 18 M 2023-11-04T05:25:33.503 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7050.g826ae420.el8 ceph-noarch 15 k 2023-11-04T05:25:33.503 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:18.0.0-7050.g826ae420.el8 ceph 24 k 2023-11-04T05:25:33.503 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-04T05:25:33.503 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-04T05:25:33.503 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-04T05:25:33.504 INFO:teuthology.orchestra.run.smithi188.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-04T05:25:33.504 INFO:teuthology.orchestra.run.smithi188.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-04T05:25:33.504 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:18.0.0-7050.g826ae420.el8 ceph 182 k 2023-11-04T05:25:33.504 INFO:teuthology.orchestra.run.smithi188.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-04T05:25:33.505 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-04T05:25:33.505 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-04T05:25:33.505 INFO:teuthology.orchestra.run.smithi188.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-04T05:25:33.505 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-04T05:25:33.505 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-04T05:25:33.505 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 518 k 2023-11-04T05:25:33.506 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-04T05:25:33.506 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:18.0.0-7050.g826ae420.el8 ceph 6.9 M 2023-11-04T05:25:33.506 INFO:teuthology.orchestra.run.smithi188.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-04T05:25:33.507 INFO:teuthology.orchestra.run.smithi188.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-04T05:25:33.507 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-04T05:25:33.507 INFO:teuthology.orchestra.run.smithi188.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-04T05:25:33.507 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-04T05:25:33.507 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-04T05:25:33.508 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-04T05:25:33.508 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-04T05:25:33.508 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-04T05:25:33.508 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-04T05:25:33.508 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-04T05:25:33.509 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-04T05:25:33.509 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-04T05:25:33.509 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-04T05:25:33.509 INFO:teuthology.orchestra.run.smithi188.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-04T05:25:33.509 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-04T05:25:33.510 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:18.0.0-7050.g826ae420.el8 ceph 45 k 2023-11-04T05:25:33.510 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:18.0.0-7050.g826ae420.el8 ceph 125 k 2023-11-04T05:25:33.510 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-04T05:25:33.510 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-04T05:25:33.510 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-04T05:25:33.511 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-04T05:25:33.511 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-04T05:25:33.511 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-04T05:25:33.511 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-04T05:25:33.511 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-04T05:25:33.512 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-04T05:25:33.512 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-04T05:25:33.512 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-04T05:25:33.512 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-04T05:25:33.512 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-04T05:25:33.513 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-04T05:25:33.513 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-04T05:25:33.513 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-04T05:25:33.513 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-04T05:25:33.513 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-04T05:25:33.514 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-04T05:25:33.514 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-04T05:25:33.514 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-04T05:25:33.514 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-04T05:25:33.514 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-04T05:25:33.515 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-04T05:25:33.515 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-04T05:25:33.515 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-04T05:25:33.515 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-04T05:25:33.515 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-04T05:25:33.516 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-04T05:25:33.516 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-04T05:25:33.516 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-04T05:25:33.516 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-04T05:25:33.516 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-04T05:25:33.517 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-04T05:25:33.517 INFO:teuthology.orchestra.run.smithi188.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-04T05:25:33.517 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-04T05:25:33.517 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-04T05:25:33.517 INFO:teuthology.orchestra.run.smithi188.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-04T05:25:33.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-04T05:25:33.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-04T05:25:33.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-04T05:25:33.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-04T05:25:33.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-04T05:25:33.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-04T05:25:33.519 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-04T05:25:33.519 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-04T05:25:33.519 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-04T05:25:33.520 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-04T05:25:33.520 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-04T05:25:33.520 INFO:teuthology.orchestra.run.smithi188.stdout:Enabling module streams: 2023-11-04T05:25:33.520 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql 10 2023-11-04T05:25:33.520 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:25:33.520 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2023-11-04T05:25:33.521 INFO:teuthology.orchestra.run.smithi188.stdout:=========================================================================================================================== 2023-11-04T05:25:33.521 INFO:teuthology.orchestra.run.smithi188.stdout:Install 128 Packages 2023-11-04T05:25:33.521 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 3 Packages 2023-11-04T05:25:33.521 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:25:33.522 INFO:teuthology.orchestra.run.smithi003.stdout:(4/131): ceph-base-18.0.0-7050.g826ae420.el8.x8 6.1 MB/s | 5.4 MB 00:00 2023-11-04T05:25:33.522 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 209 M 2023-11-04T05:25:33.523 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2023-11-04T05:25:33.769 INFO:teuthology.orchestra.run.smithi003.stdout:(5/131): ceph-mgr-18.0.0-7050.g826ae420.el8.x86 6.3 MB/s | 1.6 MB 00:00 2023-11-04T05:25:33.976 INFO:teuthology.orchestra.run.smithi188.stdout:(1/131): ceph-18.0.0-7050.g826ae420.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-11-04T05:25:34.078 INFO:teuthology.orchestra.run.smithi003.stdout:(6/131): ceph-mds-18.0.0-7050.g826ae420.el8.x86 3.6 MB/s | 2.3 MB 00:00 2023-11-04T05:25:34.344 INFO:teuthology.orchestra.run.smithi003.stdout:(7/131): ceph-common-18.0.0-7050.g826ae420.el8. 13 MB/s | 23 MB 00:01 2023-11-04T05:25:34.420 INFO:teuthology.orchestra.run.smithi003.stdout:(8/131): ceph-mon-18.0.0-7050.g826ae420.el8.x86 7.1 MB/s | 4.6 MB 00:00 2023-11-04T05:25:34.570 INFO:teuthology.orchestra.run.smithi003.stdout:(9/131): ceph-selinux-18.0.0-7050.g826ae420.el8 162 kB/s | 24 kB 00:00 2023-11-04T05:25:34.585 INFO:teuthology.orchestra.run.smithi188.stdout:(2/131): ceph-fuse-18.0.0-7050.g826ae420.el8.x8 1.5 MB/s | 919 kB 00:00 2023-11-04T05:25:35.051 INFO:teuthology.orchestra.run.smithi188.stdout:(3/131): ceph-immutable-object-cache-18.0.0-705 366 kB/s | 171 kB 00:00 2023-11-04T05:25:36.012 INFO:teuthology.orchestra.run.smithi003.stdout:(10/131): ceph-radosgw-18.0.0-7050.g826ae420.el 8.4 MB/s | 14 MB 00:01 2023-11-04T05:25:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:(11/131): cephfs-mirror-18.0.0-7050.g826ae420.e 2.8 MB/s | 262 kB 00:00 2023-11-04T05:25:36.187 INFO:teuthology.orchestra.run.smithi003.stdout:(12/131): libcephfs-devel-18.0.0-7050.g826ae420 372 kB/s | 31 kB 00:00 2023-11-04T05:25:36.354 INFO:teuthology.orchestra.run.smithi003.stdout:(13/131): libcephfs2-18.0.0-7050.g826ae420.el8. 4.9 MB/s | 839 kB 00:00 2023-11-04T05:25:36.463 INFO:teuthology.orchestra.run.smithi003.stdout:(14/131): libcephsqlite-18.0.0-7050.g826ae420.e 1.6 MB/s | 182 kB 00:00 2023-11-04T05:25:36.547 INFO:teuthology.orchestra.run.smithi003.stdout:(15/131): librados-devel-18.0.0-7050.g826ae420. 1.6 MB/s | 139 kB 00:00 2023-11-04T05:25:36.655 INFO:teuthology.orchestra.run.smithi003.stdout:(16/131): libradosstriper1-18.0.0-7050.g826ae42 4.7 MB/s | 518 kB 00:00 2023-11-04T05:25:36.877 INFO:teuthology.orchestra.run.smithi188.stdout:(4/131): ceph-base-18.0.0-7050.g826ae420.el8.x8 1.7 MB/s | 5.4 MB 00:03 2023-11-04T05:25:37.443 INFO:teuthology.orchestra.run.smithi188.stdout:(5/131): ceph-mds-18.0.0-7050.g826ae420.el8.x86 989 kB/s | 2.3 MB 00:02 2023-11-04T05:25:37.597 INFO:teuthology.orchestra.run.smithi003.stdout:(17/131): librgw2-18.0.0-7050.g826ae420.el8.x86 7.4 MB/s | 6.9 MB 00:00 2023-11-04T05:25:37.777 INFO:teuthology.orchestra.run.smithi188.stdout:(6/131): ceph-mgr-18.0.0-7050.g826ae420.el8.x86 1.8 MB/s | 1.6 MB 00:00 2023-11-04T05:25:37.793 INFO:teuthology.orchestra.run.smithi003.stdout:(18/131): ceph-osd-18.0.0-7050.g826ae420.el8.x8 4.8 MB/s | 18 MB 00:03 2023-11-04T05:25:37.814 INFO:teuthology.orchestra.run.smithi003.stdout:(19/131): python3-ceph-argparse-18.0.0-7050.g82 206 kB/s | 45 kB 00:00 2023-11-04T05:25:37.881 INFO:teuthology.orchestra.run.smithi003.stdout:(20/131): python3-ceph-common-18.0.0-7050.g826a 1.4 MB/s | 125 kB 00:00 2023-11-04T05:25:37.907 INFO:teuthology.orchestra.run.smithi003.stdout:(21/131): python3-cephfs-18.0.0-7050.g826ae420. 2.1 MB/s | 197 kB 00:00 2023-11-04T05:25:37.999 INFO:teuthology.orchestra.run.smithi003.stdout:(22/131): python3-rbd-18.0.0-7050.g826ae420.el8 3.9 MB/s | 363 kB 00:00 2023-11-04T05:25:38.024 INFO:teuthology.orchestra.run.smithi003.stdout:(23/131): python3-rados-18.0.0-7050.g826ae420.e 2.6 MB/s | 371 kB 00:00 2023-11-04T05:25:38.066 INFO:teuthology.orchestra.run.smithi003.stdout:(24/131): python3-rgw-18.0.0-7050.g826ae420.el8 1.6 MB/s | 106 kB 00:00 2023-11-04T05:25:38.108 INFO:teuthology.orchestra.run.smithi003.stdout:(25/131): rbd-fuse-18.0.0-7050.g826ae420.el8.x8 1.1 MB/s | 93 kB 00:00 2023-11-04T05:25:38.200 INFO:teuthology.orchestra.run.smithi003.stdout:(26/131): rbd-nbd-18.0.0-7050.g826ae420.el8.x86 1.9 MB/s | 180 kB 00:00 2023-11-04T05:25:38.300 INFO:teuthology.orchestra.run.smithi003.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7050.g 231 kB/s | 23 kB 00:00 2023-11-04T05:25:38.400 INFO:teuthology.orchestra.run.smithi003.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7050.g826ae42 1.3 MB/s | 132 kB 00:00 2023-11-04T05:25:38.534 INFO:teuthology.orchestra.run.smithi003.stdout:(29/131): rbd-mirror-18.0.0-7050.g826ae420.el8. 7.7 MB/s | 3.6 MB 00:00 2023-11-04T05:25:38.926 INFO:teuthology.orchestra.run.smithi003.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7050.g826ae 3.3 MB/s | 1.8 MB 00:00 2023-11-04T05:25:39.043 INFO:teuthology.orchestra.run.smithi003.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7050.g82 2.0 MB/s | 242 kB 00:00 2023-11-04T05:25:39.168 INFO:teuthology.orchestra.run.smithi003.stdout:(32/131): ceph-mgr-rook-18.0.0-7050.g826ae420.e 381 kB/s | 48 kB 00:00 2023-11-04T05:25:39.285 INFO:teuthology.orchestra.run.smithi003.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7050.g8 128 kB/s | 15 kB 00:00 2023-11-04T05:25:39.461 INFO:teuthology.orchestra.run.smithi003.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 8.0 MB/s | 7.4 MB 00:00 2023-11-04T05:25:39.486 INFO:teuthology.orchestra.run.smithi003.stdout:(35/131): ceph-volume-18.0.0-7050.g826ae420.el8 1.3 MB/s | 261 kB 00:00 2023-11-04T05:25:39.528 INFO:teuthology.orchestra.run.smithi003.stdout:(36/131): cephadm-18.0.0-7050.g826ae420.el8.noa 2.8 MB/s | 191 kB 00:00 2023-11-04T05:25:39.603 INFO:teuthology.orchestra.run.smithi003.stdout:(37/131): cephfs-top-18.0.0-7050.g826ae420.el8. 155 kB/s | 18 kB 00:00 2023-11-04T05:25:39.737 INFO:teuthology.orchestra.run.smithi003.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-11-04T05:25:39.762 INFO:teuthology.orchestra.run.smithi003.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 373 kB/s | 59 kB 00:00 2023-11-04T05:25:39.787 INFO:teuthology.orchestra.run.smithi003.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-11-04T05:25:39.821 INFO:teuthology.orchestra.run.smithi003.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-11-04T05:25:39.846 INFO:teuthology.orchestra.run.smithi003.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2023-11-04T05:25:39.871 INFO:teuthology.orchestra.run.smithi003.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-11-04T05:25:39.897 INFO:teuthology.orchestra.run.smithi003.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 656 kB/s | 33 kB 00:00 2023-11-04T05:25:39.922 INFO:teuthology.orchestra.run.smithi003.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-11-04T05:25:39.947 INFO:teuthology.orchestra.run.smithi003.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-11-04T05:25:39.981 INFO:teuthology.orchestra.run.smithi003.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2023-11-04T05:25:39.998 INFO:teuthology.orchestra.run.smithi003.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-11-04T05:25:40.023 INFO:teuthology.orchestra.run.smithi003.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-11-04T05:25:40.065 INFO:teuthology.orchestra.run.smithi003.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-11-04T05:25:40.107 INFO:teuthology.orchestra.run.smithi003.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 14 MB/s | 1.2 MB 00:00 2023-11-04T05:25:40.133 INFO:teuthology.orchestra.run.smithi003.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 728 kB/s | 49 kB 00:00 2023-11-04T05:25:40.158 INFO:teuthology.orchestra.run.smithi003.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-11-04T05:25:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-04T05:25:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-11-04T05:25:40.225 INFO:teuthology.orchestra.run.smithi003.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 374 kB/s | 16 kB 00:00 2023-11-04T05:25:40.242 INFO:teuthology.orchestra.run.smithi003.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-11-04T05:25:40.268 INFO:teuthology.orchestra.run.smithi003.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-11-04T05:25:40.293 INFO:teuthology.orchestra.run.smithi003.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-11-04T05:25:40.302 INFO:teuthology.orchestra.run.smithi188.stdout:(7/131): ceph-mon-18.0.0-7050.g826ae420.el8.x86 1.6 MB/s | 4.6 MB 00:02 2023-11-04T05:25:40.310 INFO:teuthology.orchestra.run.smithi003.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 447 kB/s | 19 kB 00:00 2023-11-04T05:25:40.327 INFO:teuthology.orchestra.run.smithi003.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 713 kB/s | 24 kB 00:00 2023-11-04T05:25:40.346 INFO:teuthology.orchestra.run.smithi003.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 875 kB/s | 29 kB 00:00 2023-11-04T05:25:40.361 INFO:teuthology.orchestra.run.smithi003.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 827 kB/s | 27 kB 00:00 2023-11-04T05:25:40.386 INFO:teuthology.orchestra.run.smithi003.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 6.0 MB/s | 255 kB 00:00 2023-11-04T05:25:40.411 INFO:teuthology.orchestra.run.smithi003.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-11-04T05:25:40.437 INFO:teuthology.orchestra.run.smithi003.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-04T05:25:40.462 INFO:teuthology.orchestra.run.smithi003.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-04T05:25:40.508 INFO:teuthology.orchestra.run.smithi003.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 323 kB/s | 23 kB 00:00 2023-11-04T05:25:40.538 INFO:teuthology.orchestra.run.smithi003.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.4 MB/s | 70 kB 00:00 2023-11-04T05:25:40.596 INFO:teuthology.orchestra.run.smithi003.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-11-04T05:25:40.621 INFO:teuthology.orchestra.run.smithi003.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 562 kB/s | 47 kB 00:00 2023-11-04T05:25:40.688 INFO:teuthology.orchestra.run.smithi003.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 734 kB/s | 43 kB 00:00 2023-11-04T05:25:40.714 INFO:teuthology.orchestra.run.smithi003.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2023-11-04T05:25:40.739 INFO:teuthology.orchestra.run.smithi003.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-04T05:25:40.773 INFO:teuthology.orchestra.run.smithi003.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 818 kB/s | 48 kB 00:00 2023-11-04T05:25:40.790 INFO:teuthology.orchestra.run.smithi003.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-04T05:25:40.823 INFO:teuthology.orchestra.run.smithi003.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 853 kB/s | 43 kB 00:00 2023-11-04T05:25:40.849 INFO:teuthology.orchestra.run.smithi003.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-11-04T05:25:40.874 INFO:teuthology.orchestra.run.smithi003.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-11-04T05:25:40.908 INFO:teuthology.orchestra.run.smithi003.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-11-04T05:25:40.933 INFO:teuthology.orchestra.run.smithi003.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-11-04T05:25:40.983 INFO:teuthology.orchestra.run.smithi003.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2023-11-04T05:25:41.017 INFO:teuthology.orchestra.run.smithi003.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 6.5 MB/s | 559 kB 00:00 2023-11-04T05:25:41.042 INFO:teuthology.orchestra.run.smithi003.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 307 kB/s | 18 kB 00:00 2023-11-04T05:25:41.076 INFO:teuthology.orchestra.run.smithi003.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-04T05:25:41.101 INFO:teuthology.orchestra.run.smithi003.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-04T05:25:41.162 INFO:teuthology.orchestra.run.smithi003.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-11-04T05:25:41.210 INFO:teuthology.orchestra.run.smithi003.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 17 MB/s | 2.3 MB 00:00 2023-11-04T05:25:41.235 INFO:teuthology.orchestra.run.smithi003.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-11-04T05:25:41.269 INFO:teuthology.orchestra.run.smithi003.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.0 MB/s | 60 kB 00:00 2023-11-04T05:25:41.286 INFO:teuthology.orchestra.run.smithi003.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-11-04T05:25:41.319 INFO:teuthology.orchestra.run.smithi003.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-04T05:25:41.345 INFO:teuthology.orchestra.run.smithi003.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 659 kB/s | 39 kB 00:00 2023-11-04T05:25:41.387 INFO:teuthology.orchestra.run.smithi003.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-11-04T05:25:41.437 INFO:teuthology.orchestra.run.smithi003.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.8 MB/s | 171 kB 00:00 2023-11-04T05:25:41.462 INFO:teuthology.orchestra.run.smithi003.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.8 MB/s | 138 kB 00:00 2023-11-04T05:25:41.504 INFO:teuthology.orchestra.run.smithi003.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-04T05:25:41.546 INFO:teuthology.orchestra.run.smithi003.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-04T05:25:41.596 INFO:teuthology.orchestra.run.smithi003.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-11-04T05:25:41.647 INFO:teuthology.orchestra.run.smithi003.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-04T05:25:41.689 INFO:teuthology.orchestra.run.smithi003.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 932 kB/s | 39 kB 00:00 2023-11-04T05:25:41.747 INFO:teuthology.orchestra.run.smithi003.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-11-04T05:25:41.798 INFO:teuthology.orchestra.run.smithi003.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-11-04T05:25:41.840 INFO:teuthology.orchestra.run.smithi003.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-04T05:25:41.999 INFO:teuthology.orchestra.run.smithi003.stdout:(105/131): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2023-11-04T05:25:42.024 INFO:teuthology.orchestra.run.smithi003.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-11-04T05:25:42.066 INFO:teuthology.orchestra.run.smithi003.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-11-04T05:25:42.099 INFO:teuthology.orchestra.run.smithi003.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 801 kB/s | 27 kB 00:00 2023-11-04T05:25:42.110 INFO:teuthology.orchestra.run.smithi188.stdout:(8/131): ceph-osd-18.0.0-7050.g826ae420.el8.x86 4.2 MB/s | 18 MB 00:04 2023-11-04T05:25:42.141 INFO:teuthology.orchestra.run.smithi003.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-04T05:25:42.152 INFO:teuthology.orchestra.run.smithi188.stdout:(9/131): ceph-selinux-18.0.0-7050.g826ae420.el8 590 kB/s | 24 kB 00:00 2023-11-04T05:25:42.175 INFO:teuthology.orchestra.run.smithi003.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-11-04T05:25:42.242 INFO:teuthology.orchestra.run.smithi003.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-11-04T05:25:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-11-04T05:25:42.342 INFO:teuthology.orchestra.run.smithi003.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2023-11-04T05:25:42.401 INFO:teuthology.orchestra.run.smithi003.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-11-04T05:25:42.426 INFO:teuthology.orchestra.run.smithi003.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2023-11-04T05:25:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-04T05:25:42.477 INFO:teuthology.orchestra.run.smithi003.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-11-04T05:25:42.502 INFO:teuthology.orchestra.run.smithi003.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-04T05:25:42.536 INFO:teuthology.orchestra.run.smithi003.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-04T05:25:42.578 INFO:teuthology.orchestra.run.smithi003.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-04T05:25:42.686 INFO:teuthology.orchestra.run.smithi003.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 26 MB/s | 4.9 MB 00:00 2023-11-04T05:25:42.728 INFO:teuthology.orchestra.run.smithi003.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-04T05:25:42.829 INFO:teuthology.orchestra.run.smithi003.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-11-04T05:25:42.862 INFO:teuthology.orchestra.run.smithi003.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-11-04T05:25:42.921 INFO:teuthology.orchestra.run.smithi003.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-11-04T05:25:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:(126/131): postgresql-server-10.21-2.module+el8 28 MB/s | 5.1 MB 00:00 2023-11-04T05:25:43.046 INFO:teuthology.orchestra.run.smithi003.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-04T05:25:43.563 INFO:teuthology.orchestra.run.smithi003.stdout:(128/131): librbd1-18.0.0-7050.g826ae420.el8.x8 7.2 MB/s | 3.7 MB 00:00 2023-11-04T05:25:43.997 INFO:teuthology.orchestra.run.smithi003.stdout:(129/131): ceph-test-18.0.0-7050.g826ae420.el8. 5.2 MB/s | 49 MB 00:09 2023-11-04T05:25:44.080 INFO:teuthology.orchestra.run.smithi003.stdout:(130/131): librados2-18.0.0-7050.g826ae420.el8. 3.5 MB/s | 3.8 MB 00:01 2023-11-04T05:25:44.131 INFO:teuthology.orchestra.run.smithi003.stdout:(131/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 1.4 MB/s | 798 kB 00:00 2023-11-04T05:25:44.135 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-11-04T05:25:44.136 INFO:teuthology.orchestra.run.smithi003.stdout:Total 18 MB/s | 209 MB 00:11 2023-11-04T05:25:44.210 INFO:teuthology.orchestra.run.smithi188.stdout:(10/131): ceph-radosgw-18.0.0-7050.g826ae420.el 3.6 MB/s | 14 MB 00:03 2023-11-04T05:25:44.327 INFO:teuthology.orchestra.run.smithi188.stdout:(11/131): cephfs-mirror-18.0.0-7050.g826ae420.e 2.2 MB/s | 262 kB 00:00 2023-11-04T05:25:44.577 INFO:teuthology.orchestra.run.smithi188.stdout:(12/131): ceph-common-18.0.0-7050.g826ae420.el8 2.1 MB/s | 23 MB 00:10 2023-11-04T05:25:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:(13/131): libcephfs-devel-18.0.0-7050.g826ae420 103 kB/s | 31 kB 00:00 2023-11-04T05:25:44.728 INFO:teuthology.orchestra.run.smithi188.stdout:(14/131): libcephsqlite-18.0.0-7050.g826ae420.e 1.8 MB/s | 182 kB 00:00 2023-11-04T05:25:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:(15/131): librados-devel-18.0.0-7050.g826ae420. 2.0 MB/s | 139 kB 00:00 2023-11-04T05:25:44.862 INFO:teuthology.orchestra.run.smithi188.stdout:(16/131): libcephfs2-18.0.0-7050.g826ae420.el8. 2.9 MB/s | 839 kB 00:00 2023-11-04T05:25:44.970 INFO:teuthology.orchestra.run.smithi188.stdout:(17/131): libradosstriper1-18.0.0-7050.g826ae42 2.9 MB/s | 518 kB 00:00 2023-11-04T05:25:44.997 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-11-04T05:25:45.038 INFO:teuthology.orchestra.run.smithi188.stdout:(18/131): python3-ceph-argparse-18.0.0-7050.g82 669 kB/s | 45 kB 00:00 2023-11-04T05:25:45.106 INFO:teuthology.orchestra.run.smithi188.stdout:(19/131): python3-ceph-common-18.0.0-7050.g826a 1.8 MB/s | 125 kB 00:00 2023-11-04T05:25:45.129 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-11-04T05:25:45.130 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-11-04T05:25:45.205 INFO:teuthology.orchestra.run.smithi188.stdout:(20/131): python3-cephfs-18.0.0-7050.g826ae420. 1.9 MB/s | 197 kB 00:00 2023-11-04T05:25:45.313 INFO:teuthology.orchestra.run.smithi188.stdout:(21/131): python3-rados-18.0.0-7050.g826ae420.e 3.4 MB/s | 371 kB 00:00 2023-11-04T05:25:45.430 INFO:teuthology.orchestra.run.smithi188.stdout:(22/131): python3-rbd-18.0.0-7050.g826ae420.el8 3.0 MB/s | 363 kB 00:00 2023-11-04T05:25:45.498 INFO:teuthology.orchestra.run.smithi188.stdout:(23/131): python3-rgw-18.0.0-7050.g826ae420.el8 1.6 MB/s | 106 kB 00:00 2023-11-04T05:25:45.556 INFO:teuthology.orchestra.run.smithi188.stdout:(24/131): rbd-fuse-18.0.0-7050.g826ae420.el8.x8 1.6 MB/s | 93 kB 00:00 2023-11-04T05:25:46.298 INFO:teuthology.orchestra.run.smithi188.stdout:(25/131): librgw2-18.0.0-7050.g826ae420.el8.x86 4.8 MB/s | 6.9 MB 00:01 2023-11-04T05:25:46.324 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-11-04T05:25:46.329 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-11-04T05:25:46.357 INFO:teuthology.orchestra.run.smithi188.stdout:(26/131): rbd-mirror-18.0.0-7050.g826ae420.el8. 4.5 MB/s | 3.6 MB 00:00 2023-11-04T05:25:46.374 INFO:teuthology.orchestra.run.smithi188.stdout:(27/131): rbd-nbd-18.0.0-7050.g826ae420.el8.x86 2.4 MB/s | 180 kB 00:00 2023-11-04T05:25:46.415 INFO:teuthology.orchestra.run.smithi188.stdout:(28/131): ceph-grafana-dashboards-18.0.0-7050.g 395 kB/s | 23 kB 00:00 2023-11-04T05:25:46.441 INFO:teuthology.orchestra.run.smithi188.stdout:(29/131): ceph-mgr-cephadm-18.0.0-7050.g826ae42 1.9 MB/s | 132 kB 00:00 2023-11-04T05:25:46.925 INFO:teuthology.orchestra.run.smithi188.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7050.g826ae 3.5 MB/s | 1.8 MB 00:00 2023-11-04T05:25:47.033 INFO:teuthology.orchestra.run.smithi188.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7050.g82 2.2 MB/s | 242 kB 00:00 2023-11-04T05:25:47.102 INFO:teuthology.orchestra.run.smithi188.stdout:(32/131): ceph-mgr-rook-18.0.0-7050.g826ae420.e 715 kB/s | 48 kB 00:00 2023-11-04T05:25:47.182 INFO:teuthology.orchestra.run.smithi188.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7050.g8 183 kB/s | 15 kB 00:00 2023-11-04T05:25:47.692 INFO:teuthology.orchestra.run.smithi188.stdout:(34/131): ceph-test-18.0.0-7050.g826ae420.el8.x 8.9 MB/s | 49 MB 00:05 2023-11-04T05:25:47.718 INFO:teuthology.orchestra.run.smithi188.stdout:(35/131): ceph-volume-18.0.0-7050.g826ae420.el8 489 kB/s | 261 kB 00:00 2023-11-04T05:25:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:(36/131): cephadm-18.0.0-7050.g826ae420.el8.noa 2.8 MB/s | 191 kB 00:00 2023-11-04T05:25:47.776 INFO:teuthology.orchestra.run.smithi188.stdout:(37/131): cephfs-top-18.0.0-7050.g826ae420.el8. 309 kB/s | 18 kB 00:00 2023-11-04T05:25:47.894 INFO:teuthology.orchestra.run.smithi188.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 5.1 MB/s | 7.4 MB 00:01 2023-11-04T05:25:47.919 INFO:teuthology.orchestra.run.smithi188.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 414 kB/s | 59 kB 00:00 2023-11-04T05:25:47.961 INFO:teuthology.orchestra.run.smithi188.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2023-11-04T05:25:48.003 INFO:teuthology.orchestra.run.smithi188.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 690 kB/s | 75 kB 00:00 2023-11-04T05:25:48.028 INFO:teuthology.orchestra.run.smithi188.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 651 kB/s | 44 kB 00:00 2023-11-04T05:25:48.053 INFO:teuthology.orchestra.run.smithi188.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-11-04T05:25:48.079 INFO:teuthology.orchestra.run.smithi188.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 656 kB/s | 33 kB 00:00 2023-11-04T05:25:48.104 INFO:teuthology.orchestra.run.smithi188.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-11-04T05:25:48.129 INFO:teuthology.orchestra.run.smithi188.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-11-04T05:25:48.155 INFO:teuthology.orchestra.run.smithi188.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-11-04T05:25:48.180 INFO:teuthology.orchestra.run.smithi188.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-11-04T05:25:48.205 INFO:teuthology.orchestra.run.smithi188.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-11-04T05:25:48.231 INFO:teuthology.orchestra.run.smithi188.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-11-04T05:25:48.256 INFO:teuthology.orchestra.run.smithi188.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-11-04T05:25:48.281 INFO:teuthology.orchestra.run.smithi188.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 967 kB/s | 49 kB 00:00 2023-11-04T05:25:48.306 INFO:teuthology.orchestra.run.smithi188.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-11-04T05:25:48.332 INFO:teuthology.orchestra.run.smithi188.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-04T05:25:48.357 INFO:teuthology.orchestra.run.smithi188.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-11-04T05:25:48.382 INFO:teuthology.orchestra.run.smithi188.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-11-04T05:25:48.408 INFO:teuthology.orchestra.run.smithi188.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-11-04T05:25:48.457 INFO:teuthology.orchestra.run.smithi188.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2023-11-04T05:25:48.483 INFO:teuthology.orchestra.run.smithi188.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 853 kB/s | 64 kB 00:00 2023-11-04T05:25:48.508 INFO:teuthology.orchestra.run.smithi188.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 364 kB/s | 19 kB 00:00 2023-11-04T05:25:48.534 INFO:teuthology.orchestra.run.smithi188.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-11-04T05:25:48.559 INFO:teuthology.orchestra.run.smithi188.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 580 kB/s | 29 kB 00:00 2023-11-04T05:25:48.585 INFO:teuthology.orchestra.run.smithi188.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 547 kB/s | 27 kB 00:00 2023-11-04T05:25:48.618 INFO:teuthology.orchestra.run.smithi188.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2023-11-04T05:25:48.643 INFO:teuthology.orchestra.run.smithi188.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-11-04T05:25:48.669 INFO:teuthology.orchestra.run.smithi188.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-04T05:25:48.694 INFO:teuthology.orchestra.run.smithi188.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.2 MB/s | 251 kB 00:00 2023-11-04T05:25:48.719 INFO:teuthology.orchestra.run.smithi188.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-11-04T05:25:48.736 INFO:teuthology.orchestra.run.smithi188.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 342 kB/s | 23 kB 00:00 2023-11-04T05:25:48.761 INFO:teuthology.orchestra.run.smithi188.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-04T05:25:48.812 INFO:teuthology.orchestra.run.smithi188.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-11-04T05:25:48.837 INFO:teuthology.orchestra.run.smithi188.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 467 kB/s | 47 kB 00:00 2023-11-04T05:25:48.879 INFO:teuthology.orchestra.run.smithi188.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-11-04T05:25:48.896 INFO:teuthology.orchestra.run.smithi188.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 543 kB/s | 43 kB 00:00 2023-11-04T05:25:48.922 INFO:teuthology.orchestra.run.smithi188.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.8 MB/s | 220 kB 00:00 2023-11-04T05:25:48.938 INFO:teuthology.orchestra.run.smithi188.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 816 kB/s | 48 kB 00:00 2023-11-04T05:25:48.955 INFO:teuthology.orchestra.run.smithi188.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-11-04T05:25:48.972 INFO:teuthology.orchestra.run.smithi188.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 846 kB/s | 43 kB 00:00 2023-11-04T05:25:48.997 INFO:teuthology.orchestra.run.smithi188.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-11-04T05:25:49.023 INFO:teuthology.orchestra.run.smithi188.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2023-11-04T05:25:49.048 INFO:teuthology.orchestra.run.smithi188.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-11-04T05:25:49.102 INFO:teuthology.orchestra.run.smithi188.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.9 MB/s | 155 kB 00:00 2023-11-04T05:25:49.140 INFO:teuthology.orchestra.run.smithi188.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-11-04T05:25:49.166 INFO:teuthology.orchestra.run.smithi188.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 282 kB/s | 18 kB 00:00 2023-11-04T05:25:49.207 INFO:teuthology.orchestra.run.smithi188.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 3.4 MB/s | 559 kB 00:00 2023-11-04T05:25:49.233 INFO:teuthology.orchestra.run.smithi188.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-04T05:25:49.258 INFO:teuthology.orchestra.run.smithi188.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-11-04T05:25:49.283 INFO:teuthology.orchestra.run.smithi188.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-11-04T05:25:49.309 INFO:teuthology.orchestra.run.smithi188.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-04T05:25:49.334 INFO:teuthology.orchestra.run.smithi188.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-11-04T05:25:49.359 INFO:teuthology.orchestra.run.smithi188.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-11-04T05:25:49.384 INFO:teuthology.orchestra.run.smithi188.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-04T05:25:49.410 INFO:teuthology.orchestra.run.smithi188.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-11-04T05:25:49.452 INFO:teuthology.orchestra.run.smithi188.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-11-04T05:25:49.469 INFO:teuthology.orchestra.run.smithi188.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-11-04T05:25:49.494 INFO:teuthology.orchestra.run.smithi188.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-11-04T05:25:49.553 INFO:teuthology.orchestra.run.smithi188.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.7 MB/s | 2.3 MB 00:00 2023-11-04T05:25:49.578 INFO:teuthology.orchestra.run.smithi188.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-11-04T05:25:49.603 INFO:teuthology.orchestra.run.smithi188.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-11-04T05:25:49.628 INFO:teuthology.orchestra.run.smithi188.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-11-04T05:25:49.662 INFO:teuthology.orchestra.run.smithi188.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-11-04T05:25:49.688 INFO:teuthology.orchestra.run.smithi188.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 660 kB/s | 39 kB 00:00 2023-11-04T05:25:49.747 INFO:teuthology.orchestra.run.smithi188.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.0 MB/s | 688 kB 00:00 2023-11-04T05:25:49.788 INFO:teuthology.orchestra.run.smithi188.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.2 MB/s | 538 kB 00:00 2023-11-04T05:25:49.813 INFO:teuthology.orchestra.run.smithi188.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-11-04T05:25:49.839 INFO:teuthology.orchestra.run.smithi188.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-11-04T05:25:49.872 INFO:teuthology.orchestra.run.smithi188.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-04T05:25:49.915 INFO:teuthology.orchestra.run.smithi188.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 639 kB/s | 27 kB 00:00 2023-11-04T05:25:49.956 INFO:teuthology.orchestra.run.smithi188.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-04T05:25:49.998 INFO:teuthology.orchestra.run.smithi188.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-11-04T05:25:50.082 INFO:teuthology.orchestra.run.smithi188.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-11-04T05:25:50.132 INFO:teuthology.orchestra.run.smithi188.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-11-04T05:25:50.174 INFO:teuthology.orchestra.run.smithi188.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-04T05:25:50.249 INFO:teuthology.orchestra.run.smithi188.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-11-04T05:25:50.266 INFO:teuthology.orchestra.run.smithi188.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-11-04T05:25:50.291 INFO:teuthology.orchestra.run.smithi188.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.9 MB/s | 126 kB 00:00 2023-11-04T05:25:50.317 INFO:teuthology.orchestra.run.smithi188.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-11-04T05:25:50.342 INFO:teuthology.orchestra.run.smithi188.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-04T05:25:50.401 INFO:teuthology.orchestra.run.smithi188.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2023-11-04T05:25:50.543 INFO:teuthology.orchestra.run.smithi188.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-11-04T05:25:50.568 INFO:teuthology.orchestra.run.smithi188.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-11-04T05:25:50.618 INFO:teuthology.orchestra.run.smithi188.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-11-04T05:25:50.668 INFO:teuthology.orchestra.run.smithi188.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-11-04T05:25:50.752 INFO:teuthology.orchestra.run.smithi188.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-11-04T05:25:50.819 INFO:teuthology.orchestra.run.smithi188.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-04T05:25:50.853 INFO:teuthology.orchestra.run.smithi188.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-11-04T05:25:50.911 INFO:teuthology.orchestra.run.smithi188.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-11-04T05:25:51.028 INFO:teuthology.orchestra.run.smithi188.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-11-04T05:25:51.087 INFO:teuthology.orchestra.run.smithi188.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-04T05:25:51.329 INFO:teuthology.orchestra.run.smithi188.stdout:(130/131): librbd1-18.0.0-7050.g826ae420.el8.x8 8.9 MB/s | 3.7 MB 00:00 2023-11-04T05:25:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-11-04T05:25:51.679 INFO:teuthology.orchestra.run.smithi188.stdout:(131/131): librados2-18.0.0-7050.g826ae420.el8. 4.6 MB/s | 3.8 MB 00:00 2023-11-04T05:25:51.681 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2023-11-04T05:25:51.682 INFO:teuthology.orchestra.run.smithi188.stdout:Total 12 MB/s | 209 MB 00:18 2023-11-04T05:25:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-04T05:25:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-04T05:25:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-04T05:25:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-04T05:25:52.516 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2023-11-04T05:25:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-04T05:25:52.635 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2023-11-04T05:25:52.636 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2023-11-04T05:25:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-04T05:25:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:18.0.0-7050.g826ae420.el8.x86_64 5/134 2023-11-04T05:25:53.674 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:18.0.0-7050.g826ae420.el8.x86_64 5/134 2023-11-04T05:25:53.770 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2023-11-04T05:25:53.774 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2023-11-04T05:25:53.866 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 6/134 2023-11-04T05:25:53.990 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 6/134 2023-11-04T05:25:54.216 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 7/134 2023-11-04T05:25:54.304 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 7/134 2023-11-04T05:25:54.454 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:18.0.0-7050.g826ae420.el8.x86_64 8/134 2023-11-04T05:25:54.660 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-04T05:25:54.899 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-04T05:25:55.128 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-04T05:25:55.551 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-04T05:25:55.838 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-04T05:25:56.032 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 14/134 2023-11-04T05:25:56.140 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 14/134 2023-11-04T05:25:56.290 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86 15/134 2023-11-04T05:25:56.443 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86 15/134 2023-11-04T05:25:56.629 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-04T05:25:56.806 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-04T05:25:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-04T05:25:57.140 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-04T05:25:57.290 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-04T05:25:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-04T05:25:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-04T05:25:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-04T05:25:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2023-11-04T05:25:58.904 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-04T05:25:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-04T05:25:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-04T05:25:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-04T05:25:59.247 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-04T05:25:59.398 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-04T05:25:59.580 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-04T05:25:59.845 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-04T05:25:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-04T05:26:00.105 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-04T05:26:00.197 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:18.0.0-7050.g826ae420.el8.x86_64 5/134 2023-11-04T05:26:00.284 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-04T05:26:00.521 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-04T05:26:00.603 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:18.0.0-7050.g826ae420.el8.x86_64 5/134 2023-11-04T05:26:00.813 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 6/134 2023-11-04T05:26:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 6/134 2023-11-04T05:26:01.138 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 7/134 2023-11-04T05:26:01.234 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 7/134 2023-11-04T05:26:01.264 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:18.0.0-7050.g826ae420.el8. 26/134 2023-11-04T05:26:01.428 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:18.0.0-7050.g826ae420.el 27/134 2023-11-04T05:26:01.451 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:18.0.0-7050.g826ae420.el8.x86_64 8/134 2023-11-04T05:26:01.654 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-04T05:26:01.664 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:18.0.0-7050.g826ae420.el8.x86_6 28/134 2023-11-04T05:26:01.865 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-04T05:26:01.885 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-04T05:26:02.066 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-04T05:26:02.460 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-04T05:26:03.013 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-04T05:26:03.204 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 14/134 2023-11-04T05:26:03.312 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 14/134 2023-11-04T05:26:03.479 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86 15/134 2023-11-04T05:26:03.632 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86 15/134 2023-11-04T05:26:03.760 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-04T05:26:03.830 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-04T05:26:04.011 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-04T05:26:04.152 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-04T05:26:04.254 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-04T05:26:04.406 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-04T05:26:04.579 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-04T05:26:04.862 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-04T05:26:05.196 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-04T05:26:05.210 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-04T05:26:05.387 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-04T05:26:05.488 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-04T05:26:05.684 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-04T05:26:05.986 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-04T05:26:06.038 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-04T05:26:06.187 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-04T05:26:06.219 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-04T05:26:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:18.0.0-7050.g826ae420.el8.x86_64 36/134 2023-11-04T05:26:06.571 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:18.0.0-7050.g826ae420.el8.x86_6 37/134 2023-11-04T05:26:06.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-04T05:26:06.884 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-04T05:26:06.900 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-04T05:26:07.022 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-04T05:26:07.096 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-04T05:26:07.214 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-04T05:26:07.339 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-04T05:26:07.402 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-04T05:26:07.519 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-04T05:26:07.597 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-04T05:26:07.680 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-04T05:26:07.765 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-04T05:26:07.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:18.0.0-7050.g826ae420.el8. 26/134 2023-11-04T05:26:07.897 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:18.0.0-7050.g826ae420.el 27/134 2023-11-04T05:26:07.989 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-04T05:26:08.106 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:18.0.0-7050.g826ae420.el8.x86_6 28/134 2023-11-04T05:26:08.177 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-04T05:26:08.339 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-04T05:26:08.533 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-04T05:26:08.628 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-04T05:26:08.708 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-04T05:26:08.844 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-04T05:26:09.025 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-04T05:26:09.236 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-04T05:26:09.751 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-04T05:26:10.050 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-04T05:26:10.269 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-04T05:26:10.400 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-04T05:26:10.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-04T05:26:10.805 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-04T05:26:11.164 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-04T05:26:11.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-04T05:26:11.456 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-04T05:26:11.627 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-04T05:26:11.724 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-04T05:26:11.858 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-04T05:26:11.979 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-04T05:26:12.052 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-04T05:26:12.212 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-04T05:26:12.226 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-04T05:26:12.380 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-04T05:26:12.554 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-04T05:26:12.563 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-04T05:26:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:18.0.0-7050.g826ae420.el8.x86_64 36/134 2023-11-04T05:26:12.862 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-04T05:26:12.908 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:18.0.0-7050.g826ae420.el8.x86_6 37/134 2023-11-04T05:26:13.051 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-04T05:26:13.080 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-04T05:26:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-04T05:26:13.340 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-04T05:26:13.352 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-04T05:26:13.452 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-04T05:26:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-04T05:26:13.652 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-04T05:26:13.656 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-04T05:26:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-04T05:26:13.967 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-04T05:26:14.060 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-04T05:26:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-04T05:26:14.221 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-04T05:26:14.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-04T05:26:14.359 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-04T05:26:14.497 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-04T05:26:14.508 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-04T05:26:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-04T05:26:14.726 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-04T05:26:14.845 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-04T05:26:14.911 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-04T05:26:15.013 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-04T05:26:15.110 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-04T05:26:15.156 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-04T05:26:15.257 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-04T05:26:15.373 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-04T05:26:15.386 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-04T05:26:15.566 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-04T05:26:15.638 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-04T05:26:15.826 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-04T05:26:15.912 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-04T05:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-04T05:26:16.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-04T05:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-04T05:26:16.371 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-04T05:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-04T05:26:16.587 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-04T05:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-04T05:26:16.823 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-04T05:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-04T05:26:17.118 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-04T05:26:17.367 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-04T05:26:17.544 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-04T05:26:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-04T05:26:17.958 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-04T05:26:18.112 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-04T05:26:18.125 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-04T05:26:18.402 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-04T05:26:18.407 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-04T05:26:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-04T05:26:18.672 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-04T05:26:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-04T05:26:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-04T05:26:19.110 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-04T05:26:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-04T05:26:19.291 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-04T05:26:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-04T05:26:19.444 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-04T05:26:19.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-04T05:26:19.932 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-04T05:26:20.088 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-04T05:26:20.235 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-04T05:26:20.260 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-04T05:26:20.399 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-04T05:26:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 92/134 2023-11-04T05:26:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 92/134 2023-11-04T05:26:20.619 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-04T05:26:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:18.0.0-7050.g826ae420.el8.x86_64 93/134 2023-11-04T05:26:20.784 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-04T05:26:20.936 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-04T05:26:21.072 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-04T05:26:21.300 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-04T05:26:21.527 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-04T05:26:21.688 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-04T05:26:21.981 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-04T05:26:22.206 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-04T05:26:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-04T05:26:22.613 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-04T05:26:23.668 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-04T05:26:23.928 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-04T05:26:24.182 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-04T05:26:24.474 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-04T05:26:24.721 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 94/134 2023-11-04T05:26:24.733 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-04T05:26:24.895 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 94/134 2023-11-04T05:26:24.936 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-04T05:26:25.505 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 94/134 2023-11-04T05:26:25.753 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-04T05:26:25.754 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 95/134 2023-11-04T05:26:25.839 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 95/134 2023-11-04T05:26:25.907 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 92/134 2023-11-04T05:26:25.963 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 92/134 2023-11-04T05:26:25.977 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 96/134 2023-11-04T05:26:26.133 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:18.0.0-7050.g826ae420.el8.x86_64 93/134 2023-11-04T05:26:29.860 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 94/134 2023-11-04T05:26:30.048 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 94/134 2023-11-04T05:26:30.631 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 94/134 2023-11-04T05:26:30.941 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 95/134 2023-11-04T05:26:31.022 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 95/134 2023-11-04T05:26:31.148 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 96/134 2023-11-04T05:26:45.460 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 96/134 2023-11-04T05:26:45.460 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2023-11-04T05:26:45.461 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2023-11-04T05:26:45.461 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-11-04T05:26:45.461 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /run 2023-11-04T05:26:45.461 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2023-11-04T05:26:45.461 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2023-11-04T05:26:45.462 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2023-11-04T05:26:45.462 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2023-11-04T05:26:45.462 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-11-04T05:26:45.462 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:26:45.825 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 97/134 2023-11-04T05:26:47.520 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 97/134 2023-11-04T05:26:47.700 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 98/134 2023-11-04T05:26:47.932 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 98/134 2023-11-04T05:26:48.100 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 99/134 2023-11-04T05:26:48.583 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 99/134 2023-11-04T05:26:48.750 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 100/134 2023-11-04T05:26:48.767 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 100/134 2023-11-04T05:26:50.036 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 96/134 2023-11-04T05:26:50.036 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2023-11-04T05:26:50.036 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2023-11-04T05:26:50.037 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /dev 2023-11-04T05:26:50.037 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /run 2023-11-04T05:26:50.037 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2023-11-04T05:26:50.037 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2023-11-04T05:26:50.038 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2023-11-04T05:26:50.038 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2023-11-04T05:26:50.038 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /dev 2023-11-04T05:26:50.038 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:26:50.352 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 97/134 2023-11-04T05:26:50.593 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:18.0.0-7050.g826ae420.el8.noarch 101/134 2023-11-04T05:26:50.786 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:18.0.0-7050.g826ae420.el8.noarch 101/134 2023-11-04T05:26:50.897 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7050.g826ae420.e 102/134 2023-11-04T05:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7050.g826ae420. 103/134 2023-11-04T05:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noa 104/134 2023-11-04T05:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noa 104/134 2023-11-04T05:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.n 105/134 2023-11-04T05:26:51.961 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 97/134 2023-11-04T05:26:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 98/134 2023-11-04T05:26:52.358 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 98/134 2023-11-04T05:26:52.544 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 99/134 2023-11-04T05:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.n 105/134 2023-11-04T05:26:53.003 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 99/134 2023-11-04T05:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7050.g826 106/134 2023-11-04T05:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7050.g826 106/134 2023-11-04T05:26:53.226 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 100/134 2023-11-04T05:26:53.243 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 100/134 2023-11-04T05:26:53.500 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7050.g826ae420.el 107/134 2023-11-04T05:26:53.691 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 108/134 2023-11-04T05:26:53.728 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 108/134 2023-11-04T05:26:53.881 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 109/134 2023-11-04T05:26:53.892 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 109/134 2023-11-04T05:26:54.871 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:18.0.0-7050.g826ae420.el8.noarch 101/134 2023-11-04T05:26:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:18.0.0-7050.g826ae420.el8.noarch 101/134 2023-11-04T05:26:55.223 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7050.g826ae420.e 102/134 2023-11-04T05:26:55.308 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:18.0.0-7050.g826ae420.el8.x86_64 110/134 2023-11-04T05:26:55.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7050.g826ae420. 103/134 2023-11-04T05:26:55.541 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 111/134 2023-11-04T05:26:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noa 104/134 2023-11-04T05:26:55.580 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 111/134 2023-11-04T05:26:55.748 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7050.g826ae 112/134 2023-11-04T05:26:55.792 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7050.g826ae 112/134 2023-11-04T05:26:55.795 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noa 104/134 2023-11-04T05:26:55.948 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 113/134 2023-11-04T05:26:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.n 105/134 2023-11-04T05:26:56.343 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 113/134 2023-11-04T05:26:56.506 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 114/134 2023-11-04T05:26:57.060 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.n 105/134 2023-11-04T05:26:57.217 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7050.g826 106/134 2023-11-04T05:26:57.308 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7050.g826 106/134 2023-11-04T05:26:57.687 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7050.g826ae420.el 107/134 2023-11-04T05:26:57.885 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 108/134 2023-11-04T05:26:57.921 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 108/134 2023-11-04T05:26:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 109/134 2023-11-04T05:26:58.111 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 109/134 2023-11-04T05:26:59.461 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:18.0.0-7050.g826ae420.el8.x86_64 110/134 2023-11-04T05:26:59.718 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 111/134 2023-11-04T05:26:59.762 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 111/134 2023-11-04T05:26:59.925 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7050.g826ae 112/134 2023-11-04T05:26:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7050.g826ae 112/134 2023-11-04T05:27:00.116 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 113/134 2023-11-04T05:27:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 113/134 2023-11-04T05:27:00.641 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 114/134 2023-11-04T05:27:01.482 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 114/134 2023-11-04T05:27:02.421 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:18.0.0-7050.g826ae420.el8.x86_64 115/134 2023-11-04T05:27:02.764 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-04T05:27:02.943 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 117/134 2023-11-04T05:27:03.196 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-04T05:27:03.338 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-04T05:27:03.542 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-04T05:27:03.751 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-04T05:27:03.960 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:18.0.0-7050.g826ae420.el8.x86_ 121/134 2023-11-04T05:27:04.098 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-04T05:27:05.336 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 114/134 2023-11-04T05:27:06.236 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:18.0.0-7050.g826ae420.el8.x86_64 115/134 2023-11-04T05:27:06.425 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-04T05:27:06.544 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-04T05:27:06.680 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-04T05:27:06.699 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-04T05:27:06.743 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 117/134 2023-11-04T05:27:06.830 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:18.0.0-7050.g826ae420.el8.noarch 124/134 2023-11-04T05:27:06.982 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-04T05:27:07.108 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 125/134 2023-11-04T05:27:07.156 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-04T05:27:07.352 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-04T05:27:07.360 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:18.0.0-7050.g826ae420.el8.x86_64 126/134 2023-11-04T05:27:07.602 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-04T05:27:07.682 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-04T05:27:07.744 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:18.0.0-7050.g826ae420.el8.x86_ 121/134 2023-11-04T05:27:07.837 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-04T05:27:07.874 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-04T05:27:07.926 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-04T05:27:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-04T05:27:08.151 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-04T05:27:08.287 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-04T05:27:08.296 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-04T05:27:08.407 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-04T05:27:08.407 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-04T05:27:08.616 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-04T05:27:08.616 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-04T05:27:08.836 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-04T05:27:09.020 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-04T05:27:09.563 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-04T05:27:09.563 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:18.0.0-7050.g826ae420.el8.x86_64 1/134 2023-11-04T05:27:09.563 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 2/134 2023-11-04T05:27:09.563 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 3/134 2023-11-04T05:27:09.564 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 4/134 2023-11-04T05:27:09.564 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7050.g826ae 5/134 2023-11-04T05:27:09.564 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 6/134 2023-11-04T05:27:09.564 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 7/134 2023-11-04T05:27:09.565 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 8/134 2023-11-04T05:27:09.565 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 9/134 2023-11-04T05:27:09.565 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 10/134 2023-11-04T05:27:09.565 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 11/134 2023-11-04T05:27:09.565 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:18.0.0-7050.g826ae420.el8.x86_64 12/134 2023-11-04T05:27:09.566 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 13/134 2023-11-04T05:27:09.566 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:18.0.0-7050.g826ae420.el8.x86_ 14/134 2023-11-04T05:27:09.566 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 15/134 2023-11-04T05:27:09.566 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 16/134 2023-11-04T05:27:09.567 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:18.0.0-7050.g826ae420.el8.x86_6 17/134 2023-11-04T05:27:09.567 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86 18/134 2023-11-04T05:27:09.567 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 19/134 2023-11-04T05:27:09.567 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7050.g826ae420.el 20/134 2023-11-04T05:27:09.567 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:18.0.0-7050.g826ae420.el8. 21/134 2023-11-04T05:27:09.568 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:18.0.0-7050.g826ae420.el8.x86_6 22/134 2023-11-04T05:27:09.568 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:18.0.0-7050.g826ae420.el8.x86_64 23/134 2023-11-04T05:27:09.568 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:18.0.0-7050.g826ae420.el8.x86_64 24/134 2023-11-04T05:27:09.568 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:18.0.0-7050.g826ae420.el8.x86_64 25/134 2023-11-04T05:27:09.568 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 26/134 2023-11-04T05:27:09.569 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 27/134 2023-11-04T05:27:09.569 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:18.0.0-7050.g826ae420.el8.x86_64 28/134 2023-11-04T05:27:09.569 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7050.g826ae420. 29/134 2023-11-04T05:27:09.569 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noa 30/134 2023-11-04T05:27:09.569 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.n 31/134 2023-11-04T05:27:09.570 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7050.g826 32/134 2023-11-04T05:27:09.570 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7050.g826ae420.el 33/134 2023-11-04T05:27:09.570 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 34/134 2023-11-04T05:27:09.570 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7050.g826ae420.e 35/134 2023-11-04T05:27:09.571 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 36/134 2023-11-04T05:27:09.571 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:18.0.0-7050.g826ae420.el8.noarch 37/134 2023-11-04T05:27:09.571 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:18.0.0-7050.g826ae420.el8.noarch 38/134 2023-11-04T05:27:09.571 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-04T05:27:09.571 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-04T05:27:09.572 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-04T05:27:09.572 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-04T05:27:09.572 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-04T05:27:09.572 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-04T05:27:09.572 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-04T05:27:09.573 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-04T05:27:09.573 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-04T05:27:09.573 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-04T05:27:09.573 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-04T05:27:09.573 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-04T05:27:09.574 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-04T05:27:09.574 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-04T05:27:09.575 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-04T05:27:09.575 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-04T05:27:09.575 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-04T05:27:09.575 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-04T05:27:09.576 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-04T05:27:09.576 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-04T05:27:09.576 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-04T05:27:09.576 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-04T05:27:09.576 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-04T05:27:09.577 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-04T05:27:09.577 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-04T05:27:09.577 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-04T05:27:09.578 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-04T05:27:09.578 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-04T05:27:09.578 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-04T05:27:09.578 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-04T05:27:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-04T05:27:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-04T05:27:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-04T05:27:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-04T05:27:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-04T05:27:09.580 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-04T05:27:09.580 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-04T05:27:09.580 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-04T05:27:09.580 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-04T05:27:09.580 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-04T05:27:09.581 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-04T05:27:09.581 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-04T05:27:09.581 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-04T05:27:09.581 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-04T05:27:09.581 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-04T05:27:09.582 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-04T05:27:09.582 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-04T05:27:09.582 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-04T05:27:09.583 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-04T05:27:09.583 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-04T05:27:09.583 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-04T05:27:09.583 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-04T05:27:09.583 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-04T05:27:09.584 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-04T05:27:09.584 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-04T05:27:09.584 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-04T05:27:09.584 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-04T05:27:09.584 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-04T05:27:09.584 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-04T05:27:09.585 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-04T05:27:09.585 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-04T05:27:09.585 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-04T05:27:09.585 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-04T05:27:09.585 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-04T05:27:09.586 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-04T05:27:09.586 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-04T05:27:09.586 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-04T05:27:09.586 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-04T05:27:09.586 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-04T05:27:09.587 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-04T05:27:09.587 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-04T05:27:09.587 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-04T05:27:09.587 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-04T05:27:09.588 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-04T05:27:09.588 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-04T05:27:09.588 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-04T05:27:09.588 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-04T05:27:09.588 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-04T05:27:09.589 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-04T05:27:09.589 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-04T05:27:09.589 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-04T05:27:09.589 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-04T05:27:09.589 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-04T05:27:09.590 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-04T05:27:09.590 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-04T05:27:09.590 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-04T05:27:09.590 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-04T05:27:09.590 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-04T05:27:09.591 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-04T05:27:09.591 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-04T05:27:09.591 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:18.0.0-7050.g826ae420.el8.x86_64 129/134 2023-11-04T05:27:09.591 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-04T05:27:09.591 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 131/134 2023-11-04T05:27:09.592 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-04T05:27:09.592 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-04T05:27:10.031 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-04T05:27:10.264 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-04T05:27:10.284 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-04T05:27:10.439 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:18.0.0-7050.g826ae420.el8.noarch 124/134 2023-11-04T05:27:10.730 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 125/134 2023-11-04T05:27:11.001 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:18.0.0-7050.g826ae420.el8.x86_64 126/134 2023-11-04T05:27:11.383 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-04T05:27:11.564 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-04T05:27:11.643 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-04T05:27:11.880 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-04T05:27:11.893 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-04T05:27:12.047 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-04T05:27:12.055 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-04T05:27:12.182 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-04T05:27:12.182 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-04T05:27:12.341 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-04T05:27:12.342 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-04T05:27:12.416 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-04T05:27:12.416 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:52 ago on Sat 04 Nov 2023 05:25:20 AM UTC. 2023-11-04T05:27:12.560 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-04T05:27:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-04T05:27:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-04T05:27:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:18.0.0-7050.g826ae420.el8.x86_64 1/134 2023-11-04T05:27:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 2/134 2023-11-04T05:27:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 3/134 2023-11-04T05:27:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 4/134 2023-11-04T05:27:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7050.g826ae 5/134 2023-11-04T05:27:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 6/134 2023-11-04T05:27:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 7/134 2023-11-04T05:27:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 8/134 2023-11-04T05:27:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 9/134 2023-11-04T05:27:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 10/134 2023-11-04T05:27:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 11/134 2023-11-04T05:27:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:18.0.0-7050.g826ae420.el8.x86_64 12/134 2023-11-04T05:27:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 13/134 2023-11-04T05:27:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:18.0.0-7050.g826ae420.el8.x86_ 14/134 2023-11-04T05:27:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 15/134 2023-11-04T05:27:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 16/134 2023-11-04T05:27:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:18.0.0-7050.g826ae420.el8.x86_6 17/134 2023-11-04T05:27:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86 18/134 2023-11-04T05:27:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 19/134 2023-11-04T05:27:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7050.g826ae420.el 20/134 2023-11-04T05:27:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:18.0.0-7050.g826ae420.el8. 21/134 2023-11-04T05:27:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:18.0.0-7050.g826ae420.el8.x86_6 22/134 2023-11-04T05:27:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:18.0.0-7050.g826ae420.el8.x86_64 23/134 2023-11-04T05:27:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:18.0.0-7050.g826ae420.el8.x86_64 24/134 2023-11-04T05:27:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:18.0.0-7050.g826ae420.el8.x86_64 25/134 2023-11-04T05:27:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 26/134 2023-11-04T05:27:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 27/134 2023-11-04T05:27:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:18.0.0-7050.g826ae420.el8.x86_64 28/134 2023-11-04T05:27:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7050.g826ae420. 29/134 2023-11-04T05:27:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noa 30/134 2023-11-04T05:27:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.n 31/134 2023-11-04T05:27:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7050.g826 32/134 2023-11-04T05:27:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7050.g826ae420.el 33/134 2023-11-04T05:27:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 34/134 2023-11-04T05:27:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7050.g826ae420.e 35/134 2023-11-04T05:27:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 36/134 2023-11-04T05:27:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:18.0.0-7050.g826ae420.el8.noarch 37/134 2023-11-04T05:27:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:18.0.0-7050.g826ae420.el8.noarch 38/134 2023-11-04T05:27:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-04T05:27:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-04T05:27:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-04T05:27:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-04T05:27:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-04T05:27:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-04T05:27:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-04T05:27:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-04T05:27:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-04T05:27:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-04T05:27:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-04T05:27:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-04T05:27:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-04T05:27:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-04T05:27:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-04T05:27:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-04T05:27:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-04T05:27:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-04T05:27:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-04T05:27:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-04T05:27:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-04T05:27:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-04T05:27:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-04T05:27:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-04T05:27:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-04T05:27:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-04T05:27:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-04T05:27:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-04T05:27:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-04T05:27:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-04T05:27:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-04T05:27:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-04T05:27:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-04T05:27:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-04T05:27:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-04T05:27:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-04T05:27:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-04T05:27:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-04T05:27:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-04T05:27:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-04T05:27:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-04T05:27:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-04T05:27:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-04T05:27:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-04T05:27:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-04T05:27:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-04T05:27:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-04T05:27:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-04T05:27:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-04T05:27:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-04T05:27:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-04T05:27:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-04T05:27:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-04T05:27:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-04T05:27:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-04T05:27:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-04T05:27:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-04T05:27:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-04T05:27:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-04T05:27:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-04T05:27:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-04T05:27:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-04T05:27:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-04T05:27:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-04T05:27:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-04T05:27:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-04T05:27:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-04T05:27:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-04T05:27:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-04T05:27:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-04T05:27:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-04T05:27:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-04T05:27:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-04T05:27:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-04T05:27:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-04T05:27:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-04T05:27:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-04T05:27:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-04T05:27:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-04T05:27:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-04T05:27:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-04T05:27:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-04T05:27:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-04T05:27:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-04T05:27:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-04T05:27:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-04T05:27:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-04T05:27:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-04T05:27:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-04T05:27:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-04T05:27:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:18.0.0-7050.g826ae420.el8.x86_64 129/134 2023-11-04T05:27:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-04T05:27:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 131/134 2023-11-04T05:27:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-04T05:27:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-04T05:27:15.063 INFO:teuthology.orchestra.run.smithi003.stdout:Installed products updated. 2023-11-04T05:27:15.299 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:15.299 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2023-11-04T05:27:15.300 INFO:teuthology.orchestra.run.smithi003.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-04T05:27:15.300 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.300 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.300 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-11-04T05:27:15.300 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-04T05:27:15.300 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.1-7.el8.noarch 2023-11-04T05:27:15.301 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.0.4-10.el8.x86_64 2023-11-04T05:27:15.301 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.301 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.301 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.301 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.302 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.302 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.302 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.302 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.302 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.303 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.303 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.303 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.303 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.303 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.304 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.304 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.304 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.304 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.304 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.305 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.305 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.305 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.305 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:15.305 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-04T05:27:15.306 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.1-9.el8.x86_64 2023-11-04T05:27:15.306 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-04T05:27:15.306 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-04T05:27:15.306 INFO:teuthology.orchestra.run.smithi003.stdout: jq-1.6-3.el8.x86_64 2023-11-04T05:27:15.306 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-04T05:27:15.307 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-04T05:27:15.307 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-04T05:27:15.307 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.307 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.307 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.308 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-04T05:27:15.308 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-04T05:27:15.308 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-04T05:27:15.308 INFO:teuthology.orchestra.run.smithi003.stdout: libpq-13.5-1.el8.x86_64 2023-11-04T05:27:15.308 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-04T05:27:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-04T05:27:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-04T05:27:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-04T05:27:15.310 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-04T05:27:15.310 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-04T05:27:15.310 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-04T05:27:15.310 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-04T05:27:15.310 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-04T05:27:15.311 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-04T05:27:15.311 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-04T05:27:15.311 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-04T05:27:15.311 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-04T05:27:15.311 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-04T05:27:15.312 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-04T05:27:15.312 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-04T05:27:15.312 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-04T05:27:15.312 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-04T05:27:15.312 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-04T05:27:15.313 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-04T05:27:15.313 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-04T05:27:15.313 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-04T05:27:15.313 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-04T05:27:15.313 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-04T05:27:15.313 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-04T05:27:15.314 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.314 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.314 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.314 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-04T05:27:15.314 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-04T05:27:15.315 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-04T05:27:15.315 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-04T05:27:15.315 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-04T05:27:15.315 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-04T05:27:15.315 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-04T05:27:15.316 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-04T05:27:15.316 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-04T05:27:15.316 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-04T05:27:15.316 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-04T05:27:15.316 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-04T05:27:15.317 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-04T05:27:15.317 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-04T05:27:15.317 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-04T05:27:15.317 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-04T05:27:15.317 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-04T05:27:15.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-04T05:27:15.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-04T05:27:15.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-04T05:27:15.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-04T05:27:15.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-04T05:27:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-2.6-1.el8.noarch 2023-11-04T05:27:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-04T05:27:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-04T05:27:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-04T05:27:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-04T05:27:15.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-04T05:27:15.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-04T05:27:15.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-04T05:27:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-04T05:27:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-04T05:27:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-04T05:27:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-04T05:27:15.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-04T05:27:15.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-04T05:27:15.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-04T05:27:15.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-04T05:27:15.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-04T05:27:15.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-04T05:27:15.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-04T05:27:15.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-04T05:27:15.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-04T05:27:15.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-04T05:27:15.323 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.324 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.324 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:15.324 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-04T05:27:15.324 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-04T05:27:15.324 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-04T05:27:15.325 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-04T05:27:15.325 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-04T05:27:15.325 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-04T05:27:15.326 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:15.326 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-11-04T05:27:15.498 DEBUG:teuthology.parallel:result is None 2023-11-04T05:27:15.967 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-04T05:27:15.968 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:54 ago on Sat 04 Nov 2023 05:25:21 AM UTC. 2023-11-04T05:27:18.672 INFO:teuthology.orchestra.run.smithi188.stdout:Installed products updated. 2023-11-04T05:27:18.902 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:18.902 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2023-11-04T05:27:18.902 INFO:teuthology.orchestra.run.smithi188.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-04T05:27:18.902 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.903 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.903 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2023-11-04T05:27:18.903 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-04T05:27:18.903 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.1-7.el8.noarch 2023-11-04T05:27:18.903 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.0.4-10.el8.x86_64 2023-11-04T05:27:18.904 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.904 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.904 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.904 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.904 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.906 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.906 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.906 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.906 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.906 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.907 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.907 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.907 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.907 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.907 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.908 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.908 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.908 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:18.0.0-7050.g826ae420.el8.noarch 2023-11-04T05:27:18.908 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-04T05:27:18.908 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.1-9.el8.x86_64 2023-11-04T05:27:18.908 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-04T05:27:18.909 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-04T05:27:18.909 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-3.el8.x86_64 2023-11-04T05:27:18.909 INFO:teuthology.orchestra.run.smithi188.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-04T05:27:18.909 INFO:teuthology.orchestra.run.smithi188.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-04T05:27:18.909 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-04T05:27:18.910 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.910 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.910 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.910 INFO:teuthology.orchestra.run.smithi188.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-04T05:27:18.910 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-04T05:27:18.911 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-04T05:27:18.911 INFO:teuthology.orchestra.run.smithi188.stdout: libpq-13.5-1.el8.x86_64 2023-11-04T05:27:18.911 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-04T05:27:18.911 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-04T05:27:18.911 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.912 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.912 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-04T05:27:18.912 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.912 INFO:teuthology.orchestra.run.smithi188.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-04T05:27:18.912 INFO:teuthology.orchestra.run.smithi188.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-04T05:27:18.912 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-04T05:27:18.913 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-04T05:27:18.913 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-04T05:27:18.913 INFO:teuthology.orchestra.run.smithi188.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-04T05:27:18.913 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-04T05:27:18.913 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-04T05:27:18.914 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-04T05:27:18.914 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-04T05:27:18.914 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-04T05:27:18.914 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-04T05:27:18.914 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-04T05:27:18.915 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-04T05:27:18.915 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-04T05:27:18.915 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-04T05:27:18.915 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-04T05:27:18.915 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-04T05:27:18.916 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-04T05:27:18.916 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-04T05:27:18.916 INFO:teuthology.orchestra.run.smithi188.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-04T05:27:18.916 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-04T05:27:18.916 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.917 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.917 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.917 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-04T05:27:18.917 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-04T05:27:18.917 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-04T05:27:18.917 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-04T05:27:18.918 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-04T05:27:18.918 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-04T05:27:18.918 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-04T05:27:18.918 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-04T05:27:18.918 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-04T05:27:18.919 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-04T05:27:18.919 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-04T05:27:18.919 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-04T05:27:18.919 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-04T05:27:18.919 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-04T05:27:18.920 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-04T05:27:18.920 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-04T05:27:18.920 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-04T05:27:18.920 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-04T05:27:18.920 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-04T05:27:18.920 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-04T05:27:18.921 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-04T05:27:18.921 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-04T05:27:18.921 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-2.6-1.el8.noarch 2023-11-04T05:27:18.921 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-04T05:27:18.921 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-04T05:27:18.922 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-04T05:27:18.922 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-04T05:27:18.922 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-04T05:27:18.922 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-04T05:27:18.922 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.922 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-04T05:27:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-04T05:27:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-04T05:27:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-04T05:27:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-04T05:27:18.924 INFO:teuthology.orchestra.run.smithi188.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-04T05:27:18.924 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-04T05:27:18.924 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-04T05:27:18.924 INFO:teuthology.orchestra.run.smithi188.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-04T05:27:18.924 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-04T05:27:18.925 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-04T05:27:18.925 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-04T05:27:18.925 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-04T05:27:18.925 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-04T05:27:18.925 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-04T05:27:18.926 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.926 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.926 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:18.0.0-7050.g826ae420.el8.x86_64 2023-11-04T05:27:18.926 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-04T05:27:18.926 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-04T05:27:18.926 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-04T05:27:18.927 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-04T05:27:18.927 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-04T05:27:18.927 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-04T05:27:18.928 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:18.928 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-11-04T05:27:19.099 DEBUG:teuthology.parallel:result is None 2023-11-04T05:27:19.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=826ae4205210243850fc3f7bc3f8aea0d1f5ced2 2023-11-04T05:27:19.334 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-04T05:27:19.363 INFO:teuthology.orchestra.run.smithi003.stdout:18.0.0-7050.g826ae420.el8 2023-11-04T05:27:19.364 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7050.g826ae420.el8 2023-11-04T05:27:19.364 INFO:teuthology.task.install:The correct ceph version 18.0.0-7050.g826ae420 is installed. 2023-11-04T05:27:19.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=826ae4205210243850fc3f7bc3f8aea0d1f5ced2 2023-11-04T05:27:19.602 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-04T05:27:19.626 INFO:teuthology.orchestra.run.smithi188.stdout:18.0.0-7050.g826ae420.el8 2023-11-04T05:27:19.627 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7050.g826ae420.el8 2023-11-04T05:27:19.627 INFO:teuthology.task.install:The correct ceph version 18.0.0-7050.g826ae420 is installed. 2023-11-04T05:27:19.628 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-04T05:27:19.628 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:19.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-04T05:27:19.679 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:19.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-04T05:27:19.721 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-04T05:27:19.722 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:19.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2023-11-04T05:27:19.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-04T05:27:19.824 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:19.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2023-11-04T05:27:19.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-04T05:27:19.917 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-04T05:27:19.918 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:19.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-04T05:27:19.948 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-04T05:27:20.018 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:20.018 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-04T05:27:20.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-04T05:27:20.112 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-04T05:27:20.112 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:20.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2023-11-04T05:27:20.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-04T05:27:20.211 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:20.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2023-11-04T05:27:20.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-04T05:27:20.306 INFO:teuthology.run_tasks:Running task ceph... 2023-11-04T05:27:20.377 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-04T05:27:20.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2023-11-04T05:27:20.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2023-11-04T05:27:20.408 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-04T05:27:20.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-04T05:27:20.451 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-04T05:27:20.543 INFO:tasks.ceph:Creating extra log directories... 2023-11-04T05:27:20.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-04T05:27:20.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-04T05:27:20.583 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-04T05:27:20.584 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-04T05:27:20.584 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-04T05:27:20.585 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:20.585 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-04T05:27:20.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-04T05:27:21.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-04T05:27:21.086 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:21.087 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-04T05:27:21.104 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-04T05:27:21.246 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-04T05:27:21.278 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-04T05:27:21.279 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-04T05:27:21.279 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447118', 'branch': 'rishabh-2023nov3', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7447118', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-2023nov3'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-2023nov3', 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9821, 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2', 'sleep_before_teardown': 0, 'subset': '332/512', 'suite': 'fs', 'suite_branch': 'rishabh-2023nov3', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_826ae4205210243850fc3f7bc3f8aea0d1f5ced2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '826ae4205210243850fc3f7bc3f8aea0d1f5ced2', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-04_04:30:51', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822'} 2023-11-04T05:27:21.280 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-04T05:27:21.283 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-04T05:27:21.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2023-11-04T05:27:21.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2023-11-04T05:27:21.397 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:21.397 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2023-11-04T05:27:21.433 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-04T05:27:21.433 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2023-11-04T05:27:21.494 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-04T05:27:21.494 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:21.495 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 388700 Links: 1 2023-11-04T05:27:21.495 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:21.495 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:21.495 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-11-04 05:27:09.239856057 +0000 2023-11-04T05:27:21.495 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-11-04 05:24:29.577005720 +0000 2023-11-04T05:27:21.496 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-11-04 05:24:29.577005720 +0000 2023-11-04T05:27:21.496 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-11-04T05:27:21.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-04T05:27:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-11-04T05:27:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-11-04T05:27:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000258002 s, 2.0 MB/s 2023-11-04T05:27:21.570 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-04T05:27:21.632 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2023-11-04T05:27:21.693 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-04T05:27:21.694 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:21.694 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 388791 Links: 1 2023-11-04T05:27:21.694 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:21.694 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:21.694 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-11-04 05:27:09.240856044 +0000 2023-11-04T05:27:21.695 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-11-04 05:24:30.100998963 +0000 2023-11-04T05:27:21.695 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-11-04 05:24:30.100998963 +0000 2023-11-04T05:27:21.695 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-11-04T05:27:21.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-04T05:27:21.765 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-11-04T05:27:21.765 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-11-04T05:27:21.765 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000213003 s, 2.4 MB/s 2023-11-04T05:27:21.766 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-04T05:27:21.827 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2023-11-04T05:27:21.887 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-04T05:27:21.887 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:21.887 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 390662 Links: 1 2023-11-04T05:27:21.888 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:21.888 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:21.888 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-11-04 05:27:09.240856044 +0000 2023-11-04T05:27:21.888 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-11-04 05:24:30.625992194 +0000 2023-11-04T05:27:21.888 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-11-04 05:24:30.625992194 +0000 2023-11-04T05:27:21.889 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-11-04T05:27:21.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-04T05:27:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-11-04T05:27:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-11-04T05:27:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00031497 s, 1.6 MB/s 2023-11-04T05:27:21.958 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-04T05:27:22.018 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2023-11-04T05:27:22.078 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-04T05:27:22.079 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:22.079 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 393111 Links: 1 2023-11-04T05:27:22.079 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:22.079 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:22.080 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-11-04 05:27:09.240856044 +0000 2023-11-04T05:27:22.080 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-11-04 05:24:31.143985514 +0000 2023-11-04T05:27:22.080 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-11-04 05:24:31.143985514 +0000 2023-11-04T05:27:22.080 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-11-04T05:27:22.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-04T05:27:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-11-04T05:27:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-11-04T05:27:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000181803 s, 2.8 MB/s 2023-11-04T05:27:22.147 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-04T05:27:22.207 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:22.207 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:22.208 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2023-11-04T05:27:22.226 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-04T05:27:22.226 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2023-11-04T05:27:22.284 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-04T05:27:22.284 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:22.284 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 389465 Links: 1 2023-11-04T05:27:22.285 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:22.285 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:22.285 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-04 05:27:12.938423457 +0000 2023-11-04T05:27:22.285 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-04 05:24:06.761822486 +0000 2023-11-04T05:27:22.285 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-04 05:24:06.761822486 +0000 2023-11-04T05:27:22.286 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-11-04T05:27:22.286 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-04T05:27:22.351 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-04T05:27:22.351 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-04T05:27:22.352 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000273463 s, 1.9 MB/s 2023-11-04T05:27:22.353 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-04T05:27:22.411 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2023-11-04T05:27:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-04T05:27:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:22.470 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 389529 Links: 1 2023-11-04T05:27:22.470 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:22.470 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:22.471 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-04 05:27:12.938423457 +0000 2023-11-04T05:27:22.471 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-04 05:24:07.285812493 +0000 2023-11-04T05:27:22.471 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-04 05:24:07.285812493 +0000 2023-11-04T05:27:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-11-04T05:27:22.472 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-04T05:27:22.535 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-04T05:27:22.535 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-04T05:27:22.536 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000223637 s, 2.3 MB/s 2023-11-04T05:27:22.537 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-04T05:27:22.594 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2023-11-04T05:27:22.651 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-04T05:27:22.652 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:22.652 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 389569 Links: 1 2023-11-04T05:27:22.652 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:22.652 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:22.652 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-04 05:27:12.938423457 +0000 2023-11-04T05:27:22.653 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-04 05:24:07.799802692 +0000 2023-11-04T05:27:22.653 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-04 05:24:07.799802692 +0000 2023-11-04T05:27:22.653 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-11-04T05:27:22.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-04T05:27:22.716 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-04T05:27:22.717 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-04T05:27:22.717 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000196178 s, 2.6 MB/s 2023-11-04T05:27:22.718 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-04T05:27:22.776 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2023-11-04T05:27:22.832 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-04T05:27:22.832 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-04T05:27:22.833 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 389610 Links: 1 2023-11-04T05:27:22.833 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-04T05:27:22.833 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-04T05:27:22.833 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-04 05:27:12.938423457 +0000 2023-11-04T05:27:22.833 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-04 05:24:08.325792662 +0000 2023-11-04T05:27:22.834 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-04 05:24:08.325792662 +0000 2023-11-04T05:27:22.834 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-11-04T05:27:22.834 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-04T05:27:22.897 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-04T05:27:22.898 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-04T05:27:22.898 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000172964 s, 3.0 MB/s 2023-11-04T05:27:22.899 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-04T05:27:22.957 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:22.957 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-04T05:27:22.957 INFO:tasks.ceph:Generating config... 2023-11-04T05:27:22.959 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-04T05:27:22.959 INFO:tasks.ceph:[client] debug client = 20 2023-11-04T05:27:22.959 INFO:tasks.ceph:[client] debug ms = 1 2023-11-04T05:27:22.959 INFO:tasks.ceph:[client] pid file = 2023-11-04T05:27:22.960 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-04T05:27:22.960 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-04T05:27:22.960 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-04T05:27:22.960 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-04T05:27:22.961 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-04T05:27:22.961 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-04T05:27:22.961 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-04T05:27:22.962 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-04T05:27:22.962 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-04T05:27:22.962 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-04T05:27:22.962 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-04T05:27:22.963 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-04T05:27:22.963 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-04T05:27:22.963 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-04T05:27:22.963 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-04T05:27:22.964 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-04T05:27:22.964 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-04T05:27:22.964 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-04T05:27:22.964 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-04T05:27:22.965 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-04T05:27:22.965 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-04T05:27:22.965 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-04T05:27:22.966 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-04T05:27:22.966 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-04T05:27:22.966 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-04T05:27:22.966 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-04T05:27:22.967 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-04T05:27:22.967 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-04T05:27:22.967 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-04T05:27:22.967 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-04T05:27:22.968 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-04T05:27:22.968 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-04T05:27:22.968 INFO:tasks.ceph:Setting up mon.a... 2023-11-04T05:27:22.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-04T05:27:23.041 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2023-11-04T05:27:23.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-04T05:27:23.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-04T05:27:23.123 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2023-11-04T05:27:23.124 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-04T05:27:23.125 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:23.125 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-04T05:27:23.181 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:23.271 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:23.272 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 7a64e299-05b0-4fd0-819f-2262a06846bb 2023-11-04T05:27:23.272 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = pacific 2023-11-04T05:27:23.272 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2023-11-04T05:27:23.272 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 7a64e299-05b0-4fd0-819f-2262a06846bb 2023-11-04T05:27:23.273 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2023-11-04T05:27:23.271086+0000 2023-11-04T05:27:23.273 INFO:teuthology.orchestra.run.smithi003.stdout:created 2023-11-04T05:27:23.271086+0000 2023-11-04T05:27:23.273 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 16 (pacific) 2023-11-04T05:27:23.273 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2023-11-04T05:27:23.273 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2023-11-04T05:27:23.274 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2023-11-04T05:27:23.274 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2023-11-04T05:27:23.274 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-04T05:27:23.275 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-04T05:27:23.292 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7a64e299-05b0-4fd0-819f-2262a06846bb... 2023-11-04T05:27:23.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-04T05:27:23.335 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-04T05:27:23.379 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-11-04T05:27:23.379 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-11-04T05:27:23.379 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-04T05:27:23.379 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-11-04T05:27:23.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.380 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-11-04T05:27:23.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.380 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-11-04T05:27:23.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.381 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-11-04T05:27:23.381 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-11-04T05:27:23.381 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.381 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-11-04T05:27:23.381 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-11-04T05:27:23.382 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.382 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-04T05:27:23.382 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-11-04T05:27:23.382 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.382 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-11-04T05:27:23.383 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-04T05:27:23.383 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.383 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-11-04T05:27:23.384 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.385 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-11-04T05:27:23.385 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-11-04T05:27:23.385 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-11-04T05:27:23.386 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-11-04T05:27:23.386 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-11-04T05:27:23.386 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-11-04T05:27:23.386 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-11-04T05:27:23.386 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-04T05:27:23.387 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-04T05:27:23.388 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-11-04T05:27:23.388 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.389 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-04T05:27:23.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.389 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-11-04T05:27:23.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.389 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-11-04T05:27:23.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.390 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-11-04T05:27:23.390 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-11-04T05:27:23.390 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-11-04T05:27:23.390 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-04T05:27:23.390 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2023-11-04T05:27:23.391 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 7a64e299-05b0-4fd0-819f-2262a06846bb 2023-11-04T05:27:23.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.391 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-11-04T05:27:23.391 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-11-04T05:27:23.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.392 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-11-04T05:27:23.392 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-11-04T05:27:23.392 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2023-11-04T05:27:23.392 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.392 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-11-04T05:27:23.392 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-11-04T05:27:23.393 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.393 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-11-04T05:27:23.393 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-11-04T05:27:23.393 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-11-04T05:27:23.393 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2023-11-04T05:27:23.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.394 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-11-04T05:27:23.394 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-11-04T05:27:23.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.394 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-11-04T05:27:23.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.395 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-11-04T05:27:23.395 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.395 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-11-04T05:27:23.395 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.395 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-11-04T05:27:23.396 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-11-04T05:27:23.396 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.396 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-11-04T05:27:23.396 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-11-04T05:27:23.396 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-11-04T05:27:23.396 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-11-04T05:27:23.397 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2023-11-04T05:27:23.397 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2023-11-04T05:27:23.397 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-11-04T05:27:23.397 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-11-04T05:27:23.398 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2023-11-04T05:27:23.398 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-11-04T05:27:23.398 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-11-04T05:27:23.398 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-11-04T05:27:23.399 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-11-04T05:27:23.399 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-11-04T05:27:23.399 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-11-04T05:27:23.399 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.399 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-11-04T05:27:23.399 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-11-04T05:27:23.400 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-11-04T05:27:23.400 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-11-04T05:27:23.400 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-11-04T05:27:23.400 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-11-04T05:27:23.400 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-11-04T05:27:23.401 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-11-04T05:27:23.401 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.401 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-11-04T05:27:23.401 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-11-04T05:27:23.401 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-11-04T05:27:23.401 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-11-04T05:27:23.402 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-11-04T05:27:23.402 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-11-04T05:27:23.402 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-11-04T05:27:23.402 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-11-04T05:27:23.402 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-11-04T05:27:23.403 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-11-04T05:27:23.403 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-11-04T05:27:23.403 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.403 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-04T05:27:23.403 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-11-04T05:27:23.404 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-11-04T05:27:23.404 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.404 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-11-04T05:27:23.404 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-04T05:27:23.404 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-04T05:27:23.404 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.405 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-11-04T05:27:23.405 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-11-04T05:27:23.405 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.405 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-11-04T05:27:23.405 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-11-04T05:27:23.406 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:27:23.406 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-11-04T05:27:23.406 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-11-04T05:27:23.406 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-11-04T05:27:23.406 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-11-04T05:27:23.406 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-04T05:27:23.407 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-04T05:27:23.407 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-11-04T05:27:23.407 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-11-04T05:27:23.407 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-11-04T05:27:23.407 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-11-04T05:27:23.408 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-11-04T05:27:23.408 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-11-04T05:27:23.408 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-11-04T05:27:23.408 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-11-04T05:27:23.408 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-11-04T05:27:23.409 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-11-04T05:27:23.409 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-11-04T05:27:23.409 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-11-04T05:27:23.409 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-11-04T05:27:23.409 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-11-04T05:27:23.409 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-11-04T05:27:23.410 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2023-11-04T05:27:23.410 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-11-04T05:27:23.410 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-11-04T05:27:23.410 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-11-04T05:27:23.410 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-11-04T05:27:23.411 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-11-04T05:27:23.411 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-11-04T05:27:23.411 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-11-04T05:27:23.411 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-11-04T05:27:23.413 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2023-11-04T05:27:23.413 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2023-11-04T05:27:23.413 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-04T05:27:23.413 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2023-11-04T05:27:23.414 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.414 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2023-11-04T05:27:23.414 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.414 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2023-11-04T05:27:23.414 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.415 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2023-11-04T05:27:23.415 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2023-11-04T05:27:23.415 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.415 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2023-11-04T05:27:23.415 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2023-11-04T05:27:23.415 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.416 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-04T05:27:23.416 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2023-11-04T05:27:23.416 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.416 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2023-11-04T05:27:23.416 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-04T05:27:23.417 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.417 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2023-11-04T05:27:23.417 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.417 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2023-11-04T05:27:23.417 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2023-11-04T05:27:23.418 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2023-11-04T05:27:23.418 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2023-11-04T05:27:23.418 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2023-11-04T05:27:23.418 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2023-11-04T05:27:23.418 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2023-11-04T05:27:23.418 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-04T05:27:23.419 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-04T05:27:23.419 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2023-11-04T05:27:23.419 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.419 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-04T05:27:23.419 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.420 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2023-11-04T05:27:23.420 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.420 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2023-11-04T05:27:23.420 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.420 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2023-11-04T05:27:23.421 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2023-11-04T05:27:23.421 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2023-11-04T05:27:23.421 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.3,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-04T05:27:23.421 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2023-11-04T05:27:23.421 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 7a64e299-05b0-4fd0-819f-2262a06846bb 2023-11-04T05:27:23.421 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.422 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2023-11-04T05:27:23.422 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2023-11-04T05:27:23.422 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.422 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2023-11-04T05:27:23.422 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2023-11-04T05:27:23.423 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2023-11-04T05:27:23.423 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.423 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2023-11-04T05:27:23.423 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2023-11-04T05:27:23.423 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.423 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2023-11-04T05:27:23.424 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2023-11-04T05:27:23.424 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2023-11-04T05:27:23.424 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2023-11-04T05:27:23.424 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.424 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2023-11-04T05:27:23.425 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2023-11-04T05:27:23.425 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.425 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2023-11-04T05:27:23.425 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.425 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2023-11-04T05:27:23.425 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.426 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2023-11-04T05:27:23.426 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.426 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2023-11-04T05:27:23.426 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2023-11-04T05:27:23.426 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.427 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2023-11-04T05:27:23.427 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2023-11-04T05:27:23.427 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2023-11-04T05:27:23.427 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2023-11-04T05:27:23.427 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2023-11-04T05:27:23.427 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2023-11-04T05:27:23.428 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-04T05:27:23.428 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2023-11-04T05:27:23.428 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2023-11-04T05:27:23.428 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2023-11-04T05:27:23.428 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2023-11-04T05:27:23.429 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2023-11-04T05:27:23.429 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2023-11-04T05:27:23.429 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2023-11-04T05:27:23.429 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-04T05:27:23.429 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.429 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2023-11-04T05:27:23.430 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-04T05:27:23.430 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2023-11-04T05:27:23.430 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-04T05:27:23.430 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-04T05:27:23.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-04T05:27:23.431 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-04T05:27:23.431 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2023-11-04T05:27:23.431 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.431 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2023-11-04T05:27:23.431 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-04T05:27:23.431 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-04T05:27:23.432 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2023-11-04T05:27:23.432 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-04T05:27:23.432 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2023-11-04T05:27:23.432 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2023-11-04T05:27:23.432 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-04T05:27:23.434 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2023-11-04T05:27:23.434 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2023-11-04T05:27:23.434 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-04T05:27:23.434 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.435 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-04T05:27:23.435 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2023-11-04T05:27:23.435 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2023-11-04T05:27:23.435 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.435 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2023-11-04T05:27:23.436 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-04T05:27:23.436 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-04T05:27:23.436 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.436 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2023-11-04T05:27:23.436 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2023-11-04T05:27:23.437 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2023-11-04T05:27:23.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2023-11-04T05:27:23.437 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-04T05:27:23.437 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2023-11-04T05:27:23.437 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2023-11-04T05:27:23.438 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2023-11-04T05:27:23.438 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2023-11-04T05:27:23.438 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-04T05:27:23.438 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-04T05:27:23.438 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2023-11-04T05:27:23.439 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2023-11-04T05:27:23.439 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-04T05:27:23.439 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2023-11-04T05:27:23.439 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-04T05:27:23.439 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-04T05:27:23.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2023-11-04T05:27:23.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2023-11-04T05:27:23.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2023-11-04T05:27:23.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-04T05:27:23.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-04T05:27:23.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-04T05:27:23.441 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-04T05:27:23.441 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2023-11-04T05:27:23.441 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2023-11-04T05:27:23.441 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2023-11-04T05:27:23.441 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-04T05:27:23.442 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2023-11-04T05:27:23.442 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2023-11-04T05:27:23.442 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2023-11-04T05:27:23.442 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2023-11-04T05:27:23.443 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-04T05:27:23.443 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-04T05:27:23.443 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-04T05:27:23.444 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-04T05:27:23.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-04T05:27:23.511 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-04T05:27:23.511 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:23.512 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-04T05:27:23.529 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:23.529 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-04T05:27:23.585 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:27:23.586 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:23.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-04T05:27:23.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-04T05:27:23.663 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:23.663 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:23.720 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:27:23.720 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:23.720 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-04T05:27:23.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-04T05:27:23.758 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:23.759 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:23.816 INFO:tasks.ceph:Setting up mon nodes... 2023-11-04T05:27:23.816 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-04T05:27:23.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-04T05:27:23.873 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-04T05:27:23.876 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-04T05:27:23.941 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-04T05:27:23.943 INFO:tasks.ceph:Setting up mds nodes... 2023-11-04T05:27:23.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-04T05:27:23.999 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-04T05:27:24.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-04T05:27:24.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-04T05:27:24.123 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-04T05:27:24.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-04T05:27:24.153 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-04T05:27:24.207 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-04T05:27:24.209 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-04T05:27:24.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-04T05:27:24.329 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-04T05:27:24.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-04T05:27:24.358 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-04T05:27:24.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-04T05:27:24.404 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-04T05:27:24.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-04T05:27:24.461 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-04T05:27:24.475 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-04T05:27:24.519 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-04T05:27:24.532 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-04T05:27:24.576 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-04T05:27:24.588 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-04T05:27:24.589 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-04T05:27:24.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-04T05:27:24.616 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:24.616 INFO:tasks.ceph:role: osd.0 2023-11-04T05:27:24.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:27:24.617 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-04T05:27:24.691 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:24.692 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:24.692 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:24.692 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:24.692 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:24.693 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:24.693 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:24.693 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:24.693 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:24.693 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:25.133 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-11-04T05:27:25.135 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-11-04T05:27:25.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-04T05:27:25.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-04T05:27:25.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-04T05:27:25.435 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:25.437 INFO:tasks.ceph:role: osd.1 2023-11-04T05:27:25.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:27:25.438 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-04T05:27:25.511 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:25.511 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:25.511 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:25.512 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:25.512 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:25.512 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:25.512 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:25.512 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:25.513 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:25.513 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:25.912 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-11-04T05:27:25.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-11-04T05:27:25.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-04T05:27:26.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-04T05:27:26.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-04T05:27:26.123 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:26.124 INFO:tasks.ceph:role: osd.2 2023-11-04T05:27:26.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:27:26.125 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-04T05:27:26.199 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:26.200 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:26.200 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:26.200 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:26.200 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:26.201 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:26.201 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:26.201 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:26.201 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:26.201 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:26.663 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-11-04T05:27:26.665 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-11-04T05:27:26.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-04T05:27:26.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-04T05:27:26.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-04T05:27:26.940 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:26.940 INFO:tasks.ceph:role: osd.3 2023-11-04T05:27:26.941 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:27:26.941 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-04T05:27:27.018 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:27.018 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:27.018 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:27.019 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:27.019 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:27.019 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:27.019 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:27.020 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:27.020 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:27.020 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:27.442 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2023-11-04T05:27:27.444 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-11-04T05:27:27.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-04T05:27:27.631 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-04T05:27:27.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:27.955+0000 7f4fb4357640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-04T05:27:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:27.955+0000 7f4fb4357640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-04T05:27:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:27.955+0000 7f4fb4357640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-04T05:27:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:27.955+0000 7f4fb4357640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-04T05:27:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:27.955+0000 7f4fb4357640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-04T05:27:27.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:27.955+0000 7f4fb4357640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-04T05:27:30.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-04T05:27:30.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:30.950+0000 7f25f148d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-04T05:27:30.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:30.950+0000 7f25f148d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-04T05:27:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:30.950+0000 7f25f148d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-04T05:27:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:30.950+0000 7f25f148d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-04T05:27:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:30.950+0000 7f25f148d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-04T05:27:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:30.950+0000 7f25f148d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-04T05:27:33.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-04T05:27:33.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:33.973+0000 7f596a9da640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-04T05:27:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:33.973+0000 7f596a9da640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-04T05:27:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:33.973+0000 7f596a9da640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-04T05:27:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:33.973+0000 7f596a9da640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-04T05:27:33.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:33.973+0000 7f596a9da640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-04T05:27:33.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:33.974+0000 7f596a9da640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-04T05:27:36.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-04T05:27:36.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:36.968+0000 7f81a8ee2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-04T05:27:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:36.969+0000 7f81a8ee2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-04T05:27:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:36.969+0000 7f81a8ee2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-04T05:27:36.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:36.969+0000 7f81a8ee2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-04T05:27:36.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:36.969+0000 7f81a8ee2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-04T05:27:36.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:36.969+0000 7f81a8ee2640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-04T05:27:39.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-04T05:27:39.863 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-04T05:27:39.897 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:39.898 INFO:tasks.ceph:role: osd.4 2023-11-04T05:27:39.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:27:39.898 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-04T05:27:39.976 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:39.976 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:39.976 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:39.976 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:39.977 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:39.977 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:39.977 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:39.977 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:39.977 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:39.977 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:40.398 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-04T05:27:40.401 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-04T05:27:40.401 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-04T05:27:40.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-04T05:27:40.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-04T05:27:40.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:40.690 INFO:tasks.ceph:role: osd.5 2023-11-04T05:27:40.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:27:40.690 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-04T05:27:40.764 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:40.765 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:40.765 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:40.765 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:40.765 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:40.765 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:40.766 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:40.766 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:40.766 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:40.766 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:41.194 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-04T05:27:41.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-04T05:27:41.196 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-04T05:27:41.314 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-04T05:27:41.346 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-04T05:27:41.414 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:41.414 INFO:tasks.ceph:role: osd.6 2023-11-04T05:27:41.414 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:27:41.415 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-04T05:27:41.489 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:41.489 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:41.489 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:41.490 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:41.490 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:41.490 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:41.490 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:41.491 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:41.491 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:41.491 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:41.934 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-04T05:27:41.936 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-04T05:27:41.936 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-04T05:27:42.107 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-04T05:27:42.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-04T05:27:42.210 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-04T05:27:42.211 INFO:tasks.ceph:role: osd.7 2023-11-04T05:27:42.211 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:27:42.212 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-04T05:27:42.286 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-04T05:27:42.287 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-04T05:27:42.287 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-04T05:27:42.287 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-04T05:27:42.287 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-04T05:27:42.288 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-04T05:27:42.288 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-04T05:27:42.288 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-04T05:27:42.288 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-04T05:27:42.288 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-04T05:27:42.729 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-04T05:27:42.731 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-04T05:27:42.731 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-04T05:27:42.904 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-04T05:27:42.939 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:43.051 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:43.049+0000 7fee80d87640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-04T05:27:43.052 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:43.050+0000 7fee80d87640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-04T05:27:43.052 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:43.050+0000 7fee80d87640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-04T05:27:43.052 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:43.050+0000 7fee80d87640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-04T05:27:43.052 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:43.050+0000 7fee80d87640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-04T05:27:43.053 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:43.050+0000 7fee80d87640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-04T05:27:45.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-04T05:27:45.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:46.047 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:46.045+0000 7f083cf87640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-04T05:27:46.047 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:46.045+0000 7f083cf87640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-04T05:27:46.048 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:46.045+0000 7f083cf87640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-04T05:27:46.048 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:46.045+0000 7f083cf87640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-04T05:27:46.048 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:46.045+0000 7f083cf87640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-04T05:27:46.048 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:46.046+0000 7f083cf87640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-04T05:27:48.914 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-04T05:27:48.947 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:49.039 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:49.037+0000 7f77bedab640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-04T05:27:49.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:49.037+0000 7f77bedab640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-04T05:27:49.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:49.037+0000 7f77bedab640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-04T05:27:49.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:49.037+0000 7f77bedab640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-04T05:27:49.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:49.037+0000 7f77bedab640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-04T05:27:49.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:49.038+0000 7f77bedab640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-04T05:27:51.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:27:51.285 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:27:51.904 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-04T05:27:51.936 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:52.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:52.026+0000 7f8c536de640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-04T05:27:52.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:52.026+0000 7f8c536de640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-04T05:27:52.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:52.026+0000 7f8c536de640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-04T05:27:52.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:52.026+0000 7f8c536de640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-04T05:27:52.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:52.026+0000 7f8c536de640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-04T05:27:52.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:52.027+0000 7f8c536de640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-04T05:27:54.901 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-04T05:27:54.933 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-04T05:27:54.934 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:54.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-04T05:27:54.967 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:54.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-04T05:27:55.038 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-04T05:27:55.110 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-04T05:27:55.180 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-04T05:27:55.251 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-04T05:27:55.320 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-04T05:27:55.389 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.390 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-04T05:27:55.422 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-04T05:27:55.493 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.493 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-04T05:27:55.563 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-04T05:27:55.633 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.633 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-04T05:27:55.701 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-04T05:27:55.770 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-04T05:27:55.837 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.837 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-04T05:27:55.858 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:55.858 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-04T05:27:55.918 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.918 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-04T05:27:55.936 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:55.937 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-04T05:27:55.994 INFO:tasks.ceph:Adding keys to all mons... 2023-11-04T05:27:55.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-04T05:27:55.997 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-04T05:27:56.039 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2023-11-04T05:27:56.039 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA71kVlYrMKNBAAFbYxO8ZiJ0xCCE1QYCt5RA== 2023-11-04T05:27:56.039 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-11-04T05:27:56.040 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA71kVlniCOOxAAiaVg5Rpzoj155dhK4fwAHQ== 2023-11-04T05:27:56.040 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-11-04T05:27:56.041 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVlmtFXBxAAOAFVrP5GVw0bydAISIXwWg== 2023-11-04T05:27:56.041 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2023-11-04T05:27:56.041 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA/1kVlv1gBORAAc9MfN39eZhn0Ev0RqWmWNw== 2023-11-04T05:27:56.042 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2023-11-04T05:27:56.042 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBC1kVlhjW2OBAATJb/Xgl1X54Q0e+dYvDaSg== 2023-11-04T05:27:56.043 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2023-11-04T05:27:56.043 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBF1kVliJUcOhAADbEH/8dZbGmzyxRzGb+JGw== 2023-11-04T05:27:56.043 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2023-11-04T05:27:56.044 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBI1kVlLb3TORAAbSvfgbvXCFbkEFqy8hSfug== 2023-11-04T05:27:56.044 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2023-11-04T05:27:56.044 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA71kVlk1EQOBAAJ26wOo3iW90PE7axk7rbCQ== 2023-11-04T05:27:56.044 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-11-04T05:27:56.045 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVlaAdTDBAADA5oMR/5/9YUnTrpYeanLg== 2023-11-04T05:27:56.045 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-11-04T05:27:56.045 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVlzqCbExAAIiPfbDlu1QpBQoFrNWdX9w== 2023-11-04T05:27:56.046 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2023-11-04T05:27:56.046 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBP1kVl0Y8LAxAAvfeWJ5QbDy+rzQtCXQ3BnA== 2023-11-04T05:27:56.047 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2023-11-04T05:27:56.047 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBS1kVlkbDDAhAA73MmMy6gIC2QbBqQd+HEBw== 2023-11-04T05:27:56.047 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2023-11-04T05:27:56.047 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBV1kVl6XVLAhAAGIbTe6PGdKGQaqFwRFFM2A== 2023-11-04T05:27:56.048 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2023-11-04T05:27:56.048 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBY1kVlxDCmARAA1mWAqF85oruCXXWnUWov7Q== 2023-11-04T05:27:56.048 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-11-04T05:27:56.048 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVl1BIWGBAAag9+v589VkQNzEcgULo1sw== 2023-11-04T05:27:56.048 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2023-11-04T05:27:56.049 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVlO86BGxAAqqXGqY9EIqyE+vWna1JW4A== 2023-11-04T05:27:56.049 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2023-11-04T05:27:56.049 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVl2LH1HhAAbFY/rBalHt4ukIO4XAtcKg== 2023-11-04T05:27:56.049 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2023-11-04T05:27:56.049 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVljlRMIhAAsld5Ip1gYgLtovVaY1oRzQ== 2023-11-04T05:27:56.061 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2023-11-04T05:27:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA71kVlYrMKNBAAFbYxO8ZiJ0xCCE1QYCt5RA== 2023-11-04T05:27:56.061 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-04T05:27:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA71kVlniCOOxAAiaVg5Rpzoj155dhK4fwAHQ== 2023-11-04T05:27:56.062 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-04T05:27:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVlmtFXBxAAOAFVrP5GVw0bydAISIXwWg== 2023-11-04T05:27:56.062 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2023-11-04T05:27:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA/1kVlv1gBORAAc9MfN39eZhn0Ev0RqWmWNw== 2023-11-04T05:27:56.063 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2023-11-04T05:27:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBC1kVlhjW2OBAATJb/Xgl1X54Q0e+dYvDaSg== 2023-11-04T05:27:56.063 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2023-11-04T05:27:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBF1kVliJUcOhAADbEH/8dZbGmzyxRzGb+JGw== 2023-11-04T05:27:56.063 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2023-11-04T05:27:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBI1kVlLb3TORAAbSvfgbvXCFbkEFqy8hSfug== 2023-11-04T05:27:56.064 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2023-11-04T05:27:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA71kVlk1EQOBAAJ26wOo3iW90PE7axk7rbCQ== 2023-11-04T05:27:56.064 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-04T05:27:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVlaAdTDBAADA5oMR/5/9YUnTrpYeanLg== 2023-11-04T05:27:56.065 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-04T05:27:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVlzqCbExAAIiPfbDlu1QpBQoFrNWdX9w== 2023-11-04T05:27:56.065 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2023-11-04T05:27:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBP1kVl0Y8LAxAAvfeWJ5QbDy+rzQtCXQ3BnA== 2023-11-04T05:27:56.065 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2023-11-04T05:27:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBS1kVlkbDDAhAA73MmMy6gIC2QbBqQd+HEBw== 2023-11-04T05:27:56.066 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2023-11-04T05:27:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBV1kVl6XVLAhAAGIbTe6PGdKGQaqFwRFFM2A== 2023-11-04T05:27:56.066 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2023-11-04T05:27:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBY1kVlxDCmARAA1mWAqF85oruCXXWnUWov7Q== 2023-11-04T05:27:56.067 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2023-11-04T05:27:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVl1BIWGBAAag9+v589VkQNzEcgULo1sw== 2023-11-04T05:27:56.067 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2023-11-04T05:27:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVlO86BGxAAqqXGqY9EIqyE+vWna1JW4A== 2023-11-04T05:27:56.067 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2023-11-04T05:27:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVl2LH1HhAAbFY/rBalHt4ukIO4XAtcKg== 2023-11-04T05:27:56.068 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2023-11-04T05:27:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQA81kVljlRMIhAAsld5Ip1gYgLtovVaY1oRzQ== 2023-11-04T05:27:56.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-04T05:27:56.093 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-04T05:27:56.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.168 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.229 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.288 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.454 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-04T05:27:56.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-04T05:27:56.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.624 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-04T05:27:56.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.793 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.848 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-04T05:27:56.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:56.905 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:56.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:56.965 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:57.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:57.024 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:57.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:57.083 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-04T05:27:57.138 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-04T05:27:57.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-04T05:27:57.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-04T05:27:57.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-04T05:27:57.292 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-04T05:27:57.319 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-04T05:27:57.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-04T05:27:57.444 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-04T05:27:57.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-04T05:27:57.608 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-04T05:27:57.635 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:57.637 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-04T05:27:57.692 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-04T05:27:57.692 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-04T05:27:57.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-04T05:27:57.696 INFO:tasks.ceph.mon.a:Started 2023-11-04T05:27:57.696 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-04T05:27:57.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-04T05:27:57.735 INFO:tasks.ceph.mon.b:Started 2023-11-04T05:27:57.735 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-04T05:27:57.736 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-04T05:27:57.738 INFO:tasks.ceph.mon.c:Started 2023-11-04T05:27:57.738 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-04T05:27:57.738 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-04T05:27:57.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-04T05:27:57.741 INFO:tasks.ceph.mgr.y:Started 2023-11-04T05:27:57.742 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-04T05:27:57.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-04T05:27:57.744 INFO:tasks.ceph.mgr.x:Started 2023-11-04T05:27:57.744 INFO:tasks.ceph:Setting crush tunables to default 2023-11-04T05:27:57.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2023-11-04T05:27:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.869+0000 7fcb0aa33700 1 Processor -- start 2023-11-04T05:27:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.869+0000 7fcb0aa33700 1 -- start start 2023-11-04T05:27:57.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb0aa33700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 0x7fcb0406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:57.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb0aa33700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcb040d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:57.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb0aa33700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb040caca0 con 0x7fcb041611d0 2023-11-04T05:27:57.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb0aa33700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040ce0a0 con 0x7fcb0406ab20 2023-11-04T05:27:57.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb0aa33700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0406b4e0 con 0x7fcb040d02d0 2023-11-04T05:27:57.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 0x7fcb0406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:57.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 0x7fcb0406afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50960/0 (socket says 172.21.15.3:50960) 2023-11-04T05:27:57.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb03fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcb041611d0 0x7fcb040ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56306/0 (socket says 172.21.15.3:56306) 2023-11-04T05:27:57.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 -- 172.21.15.3:0/968005293 learned_addr learned my addr 172.21.15.3:0/968005293 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:27:57.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 -- 172.21.15.3:0/968005293 >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 msgr2=0x7fcb0406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-04T05:27:57.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 -- 172.21.15.3:0/968005293 >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 msgr2=0x7fcb0406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-04T05:27:57.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 --2- 172.21.15.3:0/968005293 >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 0x7fcb0406afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-04T05:27:57.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcb040d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:57.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb037fe700 1 --2- 172.21.15.3:0/968005293 >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 0x7fcb0406afa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-04T05:27:57.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb08fd0700 1 -- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 msgr2=0x7fcb040d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-04T05:27:57.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb08fd0700 1 -- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 msgr2=0x7fcb040d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-04T05:27:57.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcb040d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-04T05:27:57.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:57.870+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcb040d0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-04T05:27:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.067+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855431349 0 0) 0x7fcb040caca0 con 0x7fcb041611d0 2023-11-04T05:27:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.067+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaec0035b0 con 0x7fcb041611d0 2023-11-04T05:27:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2920245143 0 0) 0x7fcaec0035b0 con 0x7fcb041611d0 2023-11-04T05:27:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 msgr2=0x7fcb040d0760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:27:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcb040d0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 msgr2=0x7fcb0406afa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:27:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 --2- 172.21.15.3:0/968005293 >> v2:172.21.15.3:3300/0 conn(0x7fcb0406ab20 0x7fcb0406afa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040cfe40 con 0x7fcb041611d0 2023-11-04T05:27:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fcaf4001670 con 0x7fcb041611d0 2023-11-04T05:27:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 >> v1:172.21.15.3:6789/0 conn(0x7fcb041611d0 legacy=0x7fcb040ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaec003a00 0x7fcaec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec004850 0x7fcaec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcaec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040ce0a0 con 0x7fcaec003a00 2023-11-04T05:27:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0406b4e0 con 0x7fcaec004850 2023-11-04T05:27:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec00baf0 con 0x7fcb040d02d0 2023-11-04T05:27:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb03fff700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaec003a00 0x7fcaec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb037fe700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec004850 0x7fcaec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcaec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb03fff700 1 -- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 msgr2=0x7fcaec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:27:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb03fff700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb040d02d0 0x7fcaec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb03fff700 1 -- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec004850 msgr2=0x7fcaec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:27:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb03fff700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec004850 0x7fcaec004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.068+0000 7fcb03fff700 1 -- 172.21.15.3:0/968005293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0035b0 con 0x7fcaec003a00 2023-11-04T05:27:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb03fff700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaec003a00 0x7fcaec003e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4007c80 tx=0x7fcaec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f1503da7d6c9a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400cbb0 con 0x7fcaec003a00 2023-11-04T05:27:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf400cd10 con 0x7fcaec003a00 2023-11-04T05:27:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb02ffd700 1 -- 172.21.15.3:0/968005293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4015b80 con 0x7fcaec003a00 2023-11-04T05:27:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb0aa33700 1 -- 172.21.15.3:0/968005293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaec003a00 msgr2=0x7fcaec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb0aa33700 1 --2- 172.21.15.3:0/968005293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaec003a00 0x7fcaec003e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4007c80 tx=0x7fcaec005880 comp rx=0 tx=0).stop 2023-11-04T05:27:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb08fd0700 1 -- 172.21.15.3:0/968005293 reap_dead start 2023-11-04T05:27:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb0aa33700 1 -- 172.21.15.3:0/968005293 shutdown_connections 2023-11-04T05:27:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb0aa33700 1 -- 172.21.15.3:0/968005293 >> 172.21.15.3:0/968005293 conn(0x7fcb04072560 msgr2=0x7fcb04099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb0aa33700 1 -- 172.21.15.3:0/968005293 shutdown_connections 2023-11-04T05:27:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.069+0000 7fcb0aa33700 1 -- 172.21.15.3:0/968005293 wait complete. 2023-11-04T05:27:58.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.071+0000 7fcb0aa33700 1 Processor -- start 2023-11-04T05:27:58.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.071+0000 7fcb0aa33700 1 -- start start 2023-11-04T05:27:58.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaec003a00 0x7fcb040c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb040ade40 0x7fcb04157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 0x7fcb04152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04153130 con 0x7fcb04152770 2023-11-04T05:27:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040ae2c0 con 0x7fcb040ade40 2023-11-04T05:27:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040c4880 con 0x7fcaec003a00 2023-11-04T05:27:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 0x7fcb04152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb03fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaec003a00 0x7fcb040c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb037fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb040ade40 0x7fcb04157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb03fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaec003a00 0x7fcb040c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:42032/0 (socket says 172.21.15.3:42032) 2023-11-04T05:27:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 0x7fcb04152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50980/0 (socket says 172.21.15.3:50980) 2023-11-04T05:27:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb03fff700 1 -- 172.21.15.3:0/2130527118 learned_addr learned my addr 172.21.15.3:0/2130527118 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:27:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb037fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb040ade40 0x7fcb04157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:50066/0 (socket says 172.21.15.3:50066) 2023-11-04T05:27:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 -- 172.21.15.3:0/2130527118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaec003a00 msgr2=0x7fcb040c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaec003a00 0x7fcb040c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 -- 172.21.15.3:0/2130527118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb040ade40 msgr2=0x7fcb04157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb040ade40 0x7fcb04157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 -- 172.21.15.3:0/2130527118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040c4b60 con 0x7fcb04152770 2023-11-04T05:27:58.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb08fd0700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 0x7fcb04152bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb04072c40 tx=0x7fcafc00f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00fcf0 con 0x7fcb04152770 2023-11-04T05:27:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00fe90 con 0x7fcb04152770 2023-11-04T05:27:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb040c4e40 con 0x7fcb04152770 2023-11-04T05:27:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.072+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc003c60 con 0x7fcb04152770 2023-11-04T05:27:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.073+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb040c85c0 con 0x7fcb04152770 2023-11-04T05:27:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.073+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcafc004030 con 0x7fcb04152770 2023-11-04T05:27:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.073+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcafc015500 con 0x7fcb04152770 2023-11-04T05:27:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.073+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb040cf390 con 0x7fcb04152770 2023-11-04T05:27:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.076+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fcafc01b070 con 0x7fcb04152770 2023-11-04T05:27:58.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.152+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcb0406de40 con 0x7fcb04152770 2023-11-04T05:27:58.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb00ff9700 1 -- 172.21.15.3:0/2130527118 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcafc004420 con 0x7fcb04152770 2023-11-04T05:27:58.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 msgr2=0x7fcb04152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb0aa33700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 0x7fcb04152bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb04072c40 tx=0x7fcafc00f430 comp rx=0 tx=0).stop 2023-11-04T05:27:58.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 shutdown_connections 2023-11-04T05:27:58.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb0aa33700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb04152770 0x7fcb04152bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb0aa33700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaec003a00 0x7fcb040c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.154+0000 7fcb0aa33700 1 --2- 172.21.15.3:0/2130527118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb040ade40 0x7fcb04157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.155+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 >> 172.21.15.3:0/2130527118 conn(0x7fcb04072560 msgr2=0x7fcb0415cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:58.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.155+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 shutdown_connections 2023-11-04T05:27:58.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:27:58.155+0000 7fcb0aa33700 1 -- 172.21.15.3:0/2130527118 wait complete. 2023-11-04T05:27:58.158 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2023-11-04T05:27:58.167 INFO:tasks.ceph:check_enable_crimson: False 2023-11-04T05:27:58.168 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-04T05:27:58.168 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:58.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-04T05:27:58.184 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:58.182+0000 7f866aa6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-04T05:27:58.196 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:58.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-04T05:27:58.250 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:58.247+0000 7f866aa6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-04T05:27:58.263 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:58.263 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-04T05:27:58.325 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:58.324+0000 7f866aa6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-04T05:27:58.330 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:27:58.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-04T05:27:58.396 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:58.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-04T05:27:58.403 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:58.402+0000 7f866aa6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-04T05:27:58.424 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:58.425 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-04T05:27:58.491 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:58.492 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-04T05:27:58.542 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:58.541+0000 7fafbdd93200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-04T05:27:58.557 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:58.555+0000 7f866aa6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-04T05:27:58.559 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:27:58.560 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-04T05:27:58.611 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:58.610+0000 7fafbdd93200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-04T05:27:58.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 384c5e00-e453-4d87-a383-a977aefa4762 0 2023-11-04T05:27:58.691 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:58.690+0000 7fafbdd93200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-04T05:27:58.773 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:58.772+0000 7fafbdd93200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-04T05:27:58.787 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.785+0000 7f5cb1858700 1 Processor -- start 2023-11-04T05:27:58.787 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.785+0000 7f5cb1858700 1 -- start start 2023-11-04T05:27:58.789 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5cb1858700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5cac15d9a0 0x7f5cac0d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.790 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5cb1858700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac0ceed0 con 0x7f5cac0d0800 2023-11-04T05:27:58.790 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5cb1858700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac0d1520 con 0x7f5cac0b2620 2023-11-04T05:27:58.790 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5cb1858700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0cf120 con 0x7f5cac15d9a0 2023-11-04T05:27:58.790 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5caa7fc700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5cac0b2620 0x7f5cac0d00f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38220/0 (socket says 172.21.15.188:38220) 2023-11-04T05:27:58.790 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5caa7fc700 1 -- 172.21.15.188:0/856679050 learned_addr learned my addr 172.21.15.188:0/856679050 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-04T05:27:58.791 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5caaffd700 1 --2- 172.21.15.188:0/856679050 >> v2:172.21.15.3:3300/0 conn(0x7f5cac15d9a0 0x7f5cac0d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.791 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5ca9ffb700 1 -- 172.21.15.188:0/856679050 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897675270 0 0) 0x7f5cac0d1520 con 0x7f5cac0b2620 2023-11-04T05:27:58.791 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5ca9ffb700 1 -- 172.21.15.188:0/856679050 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c980035b0 con 0x7f5cac0b2620 2023-11-04T05:27:58.791 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.786+0000 7f5caaffd700 1 -- 172.21.15.188:0/856679050 >> v1:172.21.15.188:6789/0 conn(0x7f5cac0b2620 legacy=0x7f5cac0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.791 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5caaffd700 1 -- 172.21.15.188:0/856679050 >> v1:172.21.15.3:6789/0 conn(0x7f5cac0d0800 legacy=0x7f5cac0d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.792 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5ca9ffb700 1 -- 172.21.15.188:0/856679050 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856264723 0 0) 0x7f5cac0ceed0 con 0x7f5cac0d0800 2023-11-04T05:27:58.792 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5caaffd700 1 -- 172.21.15.188:0/856679050 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0cf400 con 0x7f5cac15d9a0 2023-11-04T05:27:58.792 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5caaffd700 1 --2- 172.21.15.188:0/856679050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac0d3fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009840 tx=0x7f5c94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4263c9d44153612 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:58.792 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5ca9ffb700 1 -- 172.21.15.188:0/856679050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94011830 con 0x7f5cac15d9a0 2023-11-04T05:27:58.792 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5ca9ffb700 1 -- 172.21.15.188:0/856679050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94011990 con 0x7f5cac15d9a0 2023-11-04T05:27:58.793 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5ca9ffb700 1 -- 172.21.15.188:0/856679050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94011d20 con 0x7f5cac15d9a0 2023-11-04T05:27:58.793 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5cb1858700 1 -- 172.21.15.188:0/856679050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 msgr2=0x7f5cac0d3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.793 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5cb1858700 1 --2- 172.21.15.188:0/856679050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac0d3fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009840 tx=0x7f5c94009b50 comp rx=0 tx=0).stop 2023-11-04T05:27:58.793 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5cb1858700 1 -- 172.21.15.188:0/856679050 shutdown_connections 2023-11-04T05:27:58.793 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5cb1858700 1 --2- 172.21.15.188:0/856679050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac0d3fc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.794 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.787+0000 7f5cb1858700 1 -- 172.21.15.188:0/856679050 >> 172.21.15.188:0/856679050 conn(0x7f5cac08aff0 msgr2=0x7f5cac0d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:58.794 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.788+0000 7f5cb1858700 1 -- 172.21.15.188:0/856679050 shutdown_connections 2023-11-04T05:27:58.794 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.788+0000 7f5cb1858700 1 -- 172.21.15.188:0/856679050 wait complete. 2023-11-04T05:27:58.796 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.789+0000 7f5cb1858700 1 Processor -- start 2023-11-04T05:27:58.797 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.789+0000 7f5cb1858700 1 -- start start 2023-11-04T05:27:58.797 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 0x7f5cac0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.797 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 0x7f5cac06c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.797 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:58.798 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0cad30 con 0x7f5cac15d9a0 2023-11-04T05:27:58.798 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac1f34f0 con 0x7f5cac0d0800 2023-11-04T05:27:58.798 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0cdcf0 con 0x7f5cac0b2620 2023-11-04T05:27:58.798 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 0x7f5cac06c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.798 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caaffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 0x7f5cac0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.799 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 0x7f5cac06c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47150/0 (socket says 172.21.15.188:47150) 2023-11-04T05:27:58.799 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caaffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 0x7f5cac0c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36686/0 (socket says 172.21.15.188:36686) 2023-11-04T05:27:58.799 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 -- 172.21.15.188:0/2323092702 learned_addr learned my addr 172.21.15.188:0/2323092702 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:58.799 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cab7fe700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:58.799 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 -- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 msgr2=0x7f5cac0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.799 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 0x7f5cac0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.800 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 -- 172.21.15.188:0/2323092702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 msgr2=0x7f5cac06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.800 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.800 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 -- 172.21.15.188:0/2323092702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940094a0 con 0x7f5cac0d0800 2023-11-04T05:27:58.800 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caaffd700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 0x7f5cac0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:27:58.800 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5caa7fc700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 0x7f5cac06c2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c420 tx=0x7f5c9c00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:58.801 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c005d70 con 0x7f5cac0d0800 2023-11-04T05:27:58.801 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00ce30 con 0x7f5cac0d0800 2023-11-04T05:27:58.801 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0072d0 con 0x7f5cac0d0800 2023-11-04T05:27:58.801 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac0ca400 con 0x7f5cac0d0800 2023-11-04T05:27:58.801 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.790+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac1f39e0 con 0x7f5cac0d0800 2023-11-04T05:27:58.802 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.791+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c9c01c440 con 0x7f5cac0d0800 2023-11-04T05:27:58.802 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.791+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c9c012740 con 0x7f5cac0d0800 2023-11-04T05:27:58.802 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.791+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac0d0e60 con 0x7f5cac0d0800 2023-11-04T05:27:58.802 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.793+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5c9c021060 con 0x7f5cac0d0800 2023-11-04T05:27:58.880 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.878+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "384c5e00-e453-4d87-a383-a977aefa4762", "id": 0} v 0) v1 -- 0x7f5cac0b50b0 con 0x7f5cac0d0800 2023-11-04T05:27:58.886 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.884+0000 7f5ca3fff700 1 -- 172.21.15.188:0/2323092702 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "384c5e00-e453-4d87-a383-a977aefa4762", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c9c017070 con 0x7f5cac0d0800 2023-11-04T05:27:58.886 INFO:teuthology.orchestra.run.smithi188.stdout:0 2023-11-04T05:27:58.887 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 msgr2=0x7f5cac06c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:58.887 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 0x7f5cac06c2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c420 tx=0x7f5c9c00c730 comp rx=0 tx=0).stop 2023-11-04T05:27:58.888 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 shutdown_connections 2023-11-04T05:27:58.888 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cac15d9a0 0x7f5cac06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.888 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cac0b2620 0x7f5cac0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.888 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 --2- 172.21.15.188:0/2323092702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cac0d0800 0x7f5cac06c2b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:58.888 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 >> 172.21.15.188:0/2323092702 conn(0x7f5cac08aff0 msgr2=0x7f5cac0cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:58.889 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 shutdown_connections 2023-11-04T05:27:58.889 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:58.885+0000 7f5cb1858700 1 -- 172.21.15.188:0/2323092702 wait complete. 2023-11-04T05:27:58.899 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new ed1ccd37-d112-48a3-97a6-0d1be873a282 1 2023-11-04T05:27:58.923 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:58.922+0000 7fafbdd93200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-04T05:27:58.962 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:58.960+0000 7f866aa6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-04T05:27:59.021 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.019+0000 7f4c50f5d700 1 Processor -- start 2023-11-04T05:27:59.021 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.019+0000 7f4c50f5d700 1 -- start start 2023-11-04T05:27:59.024 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c50f5d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c4c0cc100 0x7f4c4c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.024 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c50f5d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c0b0f20 con 0x7f4c4c1601a0 2023-11-04T05:27:59.024 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c50f5d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c15f4a0 con 0x7f4c4c155df0 2023-11-04T05:27:59.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c50f5d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c15c560 con 0x7f4c4c0cc100 2023-11-04T05:27:59.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c4a59c700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4c4c155df0 0x7f4c4c15be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38230/0 (socket says 172.21.15.188:38230) 2023-11-04T05:27:59.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c4a59c700 1 -- 172.21.15.188:0/2284590335 learned_addr learned my addr 172.21.15.188:0/2284590335 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-04T05:27:59.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c49d9b700 1 --2- 172.21.15.188:0/2284590335 >> v2:172.21.15.3:3300/0 conn(0x7f4c4c0cc100 0x7f4c4c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034974025 0 0) 0x7f4c4c15f4a0 con 0x7f4c4c155df0 2023-11-04T05:27:59.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.020+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c380035b0 con 0x7f4c4c155df0 2023-11-04T05:27:59.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102693894 0 0) 0x7f4c44001490 con 0x7f4c4c1601a0 2023-11-04T05:27:59.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c15f4a0 con 0x7f4c4c1601a0 2023-11-04T05:27:59.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501163963 0 0) 0x7f4c380035b0 con 0x7f4c4c155df0 2023-11-04T05:27:59.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c49d9b700 1 -- 172.21.15.188:0/2284590335 >> v1:172.21.15.188:6789/0 conn(0x7f4c4c155df0 legacy=0x7f4c4c15be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c49d9b700 1 -- 172.21.15.188:0/2284590335 >> v1:172.21.15.3:6789/0 conn(0x7f4c4c1601a0 legacy=0x7f4c4c06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c49d9b700 1 -- 172.21.15.188:0/2284590335 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c15c770 con 0x7f4c4c0cc100 2023-11-04T05:27:59.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c49d9b700 1 --2- 172.21.15.188:0/2284590335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c15fc60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0097c0 tx=0x7f4c3c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2dbb99f8396b9d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34001670 con 0x7f4c4c0cc100 2023-11-04T05:27:59.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c004030 con 0x7f4c4c0cc100 2023-11-04T05:27:59.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.021+0000 7f4c4959a700 1 -- 172.21.15.188:0/2284590335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c0043c0 con 0x7f4c4c0cc100 2023-11-04T05:27:59.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 -- 172.21.15.188:0/2284590335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 msgr2=0x7f4c4c15fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 --2- 172.21.15.188:0/2284590335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c15fc60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0097c0 tx=0x7f4c3c009ad0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 -- 172.21.15.188:0/2284590335 shutdown_connections 2023-11-04T05:27:59.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 --2- 172.21.15.188:0/2284590335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c15fc60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.028 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 -- 172.21.15.188:0/2284590335 >> 172.21.15.188:0/2284590335 conn(0x7f4c4c072560 msgr2=0x7f4c4c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 -- 172.21.15.188:0/2284590335 shutdown_connections 2023-11-04T05:27:59.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.022+0000 7f4c50f5d700 1 -- 172.21.15.188:0/2284590335 wait complete. 2023-11-04T05:27:59.031 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.023+0000 7f4c50f5d700 1 Processor -- start 2023-11-04T05:27:59.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.023+0000 7f4c50f5d700 1 -- start start 2023-11-04T05:27:59.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c50f5d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c50f5d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 0x7f4c4c0c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c50f5d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0c5100 0x7f4c4c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c50f5d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1f7440 con 0x7f4c4c0cc100 2023-11-04T05:27:59.033 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c50f5d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c14e090 con 0x7f4c4c0c5100 2023-11-04T05:27:59.033 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c50f5d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1f6ac0 con 0x7f4c4c155df0 2023-11-04T05:27:59.033 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c49d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 0x7f4c4c0c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.033 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4ad9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0c5100 0x7f4c4c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.033 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c49d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 0x7f4c4c0c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36700/0 (socket says 172.21.15.188:36700) 2023-11-04T05:27:59.034 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c49d9b700 1 -- 172.21.15.188:0/1776514578 learned_addr learned my addr 172.21.15.188:0/1776514578 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:59.034 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4a59c700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.034 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4ad9d700 1 -- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 msgr2=0x7f4c4c0c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.034 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4ad9d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 0x7f4c4c0c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4ad9d700 1 -- 172.21.15.188:0/1776514578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 msgr2=0x7f4c4c14db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4ad9d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c14db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.024+0000 7f4c4ad9d700 1 -- 172.21.15.188:0/1776514578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0094a0 con 0x7f4c4c0c5100 2023-11-04T05:27:59.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c49d9b700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 0x7f4c4c0c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:27:59.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c4ad9d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0c5100 0x7f4c4c15e0f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400cbb0 tx=0x7f4c4400cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44003c00 con 0x7f4c4c0c5100 2023-11-04T05:27:59.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44003d60 con 0x7f4c4c0c5100 2023-11-04T05:27:59.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400aae0 con 0x7f4c4c0c5100 2023-11-04T05:27:59.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c0c5610 con 0x7f4c4c0c5100 2023-11-04T05:27:59.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c1f7060 con 0x7f4c4c0c5100 2023-11-04T05:27:59.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4c440050d0 con 0x7f4c4c0c5100 2023-11-04T05:27:59.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.025+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f4c44012640 con 0x7f4c4c0c5100 2023-11-04T05:27:59.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.026+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c0b6440 con 0x7f4c4c0c5100 2023-11-04T05:27:59.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.028+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4c4c0b0f20 con 0x7f4c4c0c5100 2023-11-04T05:27:59.087 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:59.085+0000 7f866aa6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-04T05:27:59.116 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.114+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ed1ccd37-d112-48a3-97a6-0d1be873a282", "id": 1} v 0) v1 -- 0x7f4c4c1f7580 con 0x7f4c4c0c5100 2023-11-04T05:27:59.124 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.122+0000 7f4c42ffd700 1 -- 172.21.15.188:0/1776514578 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed1ccd37-d112-48a3-97a6-0d1be873a282", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c4c1f7580 con 0x7f4c4c0c5100 2023-11-04T05:27:59.124 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-04T05:27:59.125 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0c5100 msgr2=0x7f4c4c15e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0c5100 0x7f4c4c15e0f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400cbb0 tx=0x7f4c4400cec0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 shutdown_connections 2023-11-04T05:27:59.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0c5100 0x7f4c4c15e0f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c4c0cc100 0x7f4c4c14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 --2- 172.21.15.188:0/1776514578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c4c155df0 0x7f4c4c0c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.127 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 >> 172.21.15.188:0/1776514578 conn(0x7f4c4c072560 msgr2=0x7f4c4c06ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.127 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 shutdown_connections 2023-11-04T05:27:59.127 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.123+0000 7f4c50f5d700 1 -- 172.21.15.188:0/1776514578 wait complete. 2023-11-04T05:27:59.137 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new a31bc146-fd57-4ad8-9f9b-3a4736a7b582 2 2023-11-04T05:27:59.179 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:59.177+0000 7f866aa6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-04T05:27:59.260 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.258+0000 7ff4e791a700 1 Processor -- start 2023-11-04T05:27:59.260 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.258+0000 7ff4e791a700 1 -- start start 2023-11-04T05:27:59.262 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e791a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4e00cc100 0x7ff4e015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e791a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4e01601a0 0x7ff4e006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e791a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e00b0f20 con 0x7ff4e0155df0 2023-11-04T05:27:59.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e791a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e00cc590 con 0x7ff4e01601a0 2023-11-04T05:27:59.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e791a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e015c560 con 0x7ff4e00cc100 2023-11-04T05:27:59.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4e00cc100 0x7ff4e015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4e00cc100 0x7ff4e015fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47168/0 (socket says 172.21.15.188:47168) 2023-11-04T05:27:59.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 -- 172.21.15.188:0/1335663668 learned_addr learned my addr 172.21.15.188:0/1335663668 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:59.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e5eb7700 1 --2- 172.21.15.188:0/1335663668 >> v2:172.21.15.3:3300/0 conn(0x7ff4e01601a0 0x7ff4e006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 -- 172.21.15.188:0/1335663668 >> v2:172.21.15.3:3300/0 conn(0x7ff4e01601a0 msgr2=0x7ff4e006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:27:59.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 --2- 172.21.15.188:0/1335663668 >> v2:172.21.15.3:3300/0 conn(0x7ff4e01601a0 0x7ff4e006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 -- 172.21.15.188:0/1335663668 >> v1:172.21.15.3:6789/0 conn(0x7ff4e0155df0 legacy=0x7ff4e015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 -- 172.21.15.188:0/1335663668 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e015f4a0 con 0x7ff4e00cc100 2023-11-04T05:27:59.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.259+0000 7ff4e4eb5700 1 --2- 172.21.15.188:0/1335663668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00cc100 0x7ff4e015fc60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00c76d0 tx=0x7ff4d4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17453bd827a36f9b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4dffff700 1 -- 172.21.15.188:0/1335663668 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4011830 con 0x7ff4e00cc100 2023-11-04T05:27:59.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4dffff700 1 -- 172.21.15.188:0/1335663668 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d4011990 con 0x7ff4e00cc100 2023-11-04T05:27:59.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4dffff700 1 -- 172.21.15.188:0/1335663668 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4011d20 con 0x7ff4e00cc100 2023-11-04T05:27:59.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 -- 172.21.15.188:0/1335663668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00cc100 msgr2=0x7ff4e015fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1335663668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00cc100 0x7ff4e015fc60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00c76d0 tx=0x7ff4d4009bd0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 -- 172.21.15.188:0/1335663668 shutdown_connections 2023-11-04T05:27:59.267 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1335663668 >> v2:172.21.15.3:3300/0 conn(0x7ff4e01601a0 0x7ff4e006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.267 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1335663668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00cc100 0x7ff4e015fc60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.267 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 -- 172.21.15.188:0/1335663668 >> 172.21.15.188:0/1335663668 conn(0x7ff4e0072560 msgr2=0x7ff4e0099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.267 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 -- 172.21.15.188:0/1335663668 shutdown_connections 2023-11-04T05:27:59.267 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.260+0000 7ff4e791a700 1 -- 172.21.15.188:0/1335663668 wait complete. 2023-11-04T05:27:59.270 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.261+0000 7ff4e791a700 1 Processor -- start 2023-11-04T05:27:59.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 -- start start 2023-11-04T05:27:59.274 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 0x7ff4e00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.274 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4e0155df0 0x7ff4e00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.274 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 0x7ff4e00af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.274 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00ae9b0 con 0x7ff4e0155df0 2023-11-04T05:27:59.274 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00c6080 con 0x7ff4e00b4810 2023-11-04T05:27:59.275 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e791a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00c4320 con 0x7ff4e00cc100 2023-11-04T05:27:59.275 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e56b6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 0x7ff4e00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.275 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e5eb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 0x7ff4e00af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.275 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e56b6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 0x7ff4e00b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36710/0 (socket says 172.21.15.188:36710) 2023-11-04T05:27:59.275 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e5eb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 0x7ff4e00af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47176/0 (socket says 172.21.15.188:47176) 2023-11-04T05:27:59.276 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e56b6700 1 -- 172.21.15.188:0/1866721764 learned_addr learned my addr 172.21.15.188:0/1866721764 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:59.276 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.262+0000 7ff4e4eb5700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4e0155df0 0x7ff4e00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.276 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e5eb7700 1 -- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 msgr2=0x7ff4e00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.276 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e5eb7700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 0x7ff4e00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.276 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e5eb7700 1 -- 172.21.15.188:0/1866721764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4e0155df0 msgr2=0x7ff4e00af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.277 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e5eb7700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4e0155df0 0x7ff4e00af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.277 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e5eb7700 1 -- 172.21.15.188:0/1866721764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4009550 con 0x7ff4e00b4810 2023-11-04T05:27:59.277 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e56b6700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 0x7ff4e00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:27:59.277 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e5eb7700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 0x7ff4e00af9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ebd0 tx=0x7ff4d000eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.277 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000ca00 con 0x7ff4e00b4810 2023-11-04T05:27:59.277 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d000f040 con 0x7ff4e00b4810 2023-11-04T05:27:59.278 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000cda0 con 0x7ff4e00b4810 2023-11-04T05:27:59.278 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e00c4600 con 0x7ff4e00b4810 2023-11-04T05:27:59.278 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e00c4bc0 con 0x7ff4e00b4810 2023-11-04T05:27:59.278 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff4d0014780 con 0x7ff4e00b4810 2023-11-04T05:27:59.278 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.263+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7ff4d001db90 con 0x7ff4e00b4810 2023-11-04T05:27:59.279 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.264+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e00c4de0 con 0x7ff4e00b4810 2023-11-04T05:27:59.279 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.266+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff4d003bce0 con 0x7ff4e00b4810 2023-11-04T05:27:59.279 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:59.268+0000 7f866aa6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-04T05:27:59.334 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:59.333+0000 7fafbdd93200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-04T05:27:59.353 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.351+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a31bc146-fd57-4ad8-9f9b-3a4736a7b582", "id": 2} v 0) v1 -- 0x7ff4e00c6fb0 con 0x7ff4e00b4810 2023-11-04T05:27:59.358 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.357+0000 7ff4ddffb700 1 -- 172.21.15.188:0/1866721764 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a31bc146-fd57-4ad8-9f9b-3a4736a7b582", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff4d002a070 con 0x7ff4e00b4810 2023-11-04T05:27:59.358 INFO:teuthology.orchestra.run.smithi188.stdout:2 2023-11-04T05:27:59.360 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.357+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 msgr2=0x7ff4e00af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.360 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.357+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 0x7ff4e00af9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ebd0 tx=0x7ff4d000eee0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.360 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 shutdown_connections 2023-11-04T05:27:59.360 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4e00b4810 0x7ff4e00af9d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.360 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4e00cc100 0x7ff4e00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 --2- 172.21.15.188:0/1866721764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4e0155df0 0x7ff4e00af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 >> 172.21.15.188:0/1866721764 conn(0x7ff4e0072560 msgr2=0x7ff4e00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 shutdown_connections 2023-11-04T05:27:59.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.358+0000 7ff4e791a700 1 -- 172.21.15.188:0/1866721764 wait complete. 2023-11-04T05:27:59.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f8a58fb6-14ea-4604-9d75-0d1b237cad96 3 2023-11-04T05:27:59.456 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:59.455+0000 7fafbdd93200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-04T05:27:59.533 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.531+0000 7f7e444cd700 1 Processor -- start 2023-11-04T05:27:59.533 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.531+0000 7f7e444cd700 1 -- start start 2023-11-04T05:27:59.536 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e444cd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e3c155df0 0x7f7e3c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.536 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e444cd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e3c15c2e0 0x7f7e3c06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.536 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e444cd700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c0b0f20 con 0x7f7e3c15be50 2023-11-04T05:27:59.536 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e444cd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c15f4a0 con 0x7f7e3c15c2e0 2023-11-04T05:27:59.536 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e444cd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c160370 con 0x7f7e3c155df0 2023-11-04T05:27:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42269700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e3c155df0 0x7f7e3c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e41a68700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7e3c15be50 0x7f7e3c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38254/0 (socket says 172.21.15.188:38254) 2023-11-04T05:27:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e41a68700 1 -- 172.21.15.188:0/2945980186 learned_addr learned my addr 172.21.15.188:0/2945980186 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-04T05:27:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42a6a700 1 --2- 172.21.15.188:0/2945980186 >> v2:172.21.15.3:3300/0 conn(0x7f7e3c15c2e0 0x7f7e3c06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42269700 1 -- 172.21.15.188:0/2945980186 >> v2:172.21.15.3:3300/0 conn(0x7f7e3c15c2e0 msgr2=0x7f7e3c06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:27:59.538 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42269700 1 --2- 172.21.15.188:0/2945980186 >> v2:172.21.15.3:3300/0 conn(0x7f7e3c15c2e0 0x7f7e3c06ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.538 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42269700 1 -- 172.21.15.188:0/2945980186 >> v1:172.21.15.188:6789/0 conn(0x7f7e3c15be50 legacy=0x7f7e3c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.538 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42269700 1 -- 172.21.15.188:0/2945980186 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c15c770 con 0x7f7e3c155df0 2023-11-04T05:27:59.538 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e41267700 1 -- 172.21.15.188:0/2945980186 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520611901 0 0) 0x7f7e2c001110 con 0x7f7e3c15be50 2023-11-04T05:27:59.538 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.533+0000 7f7e42269700 1 --2- 172.21.15.188:0/2945980186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0cc100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c09e710 tx=0x7f7e38009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c87eeca4e7e0b8c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.534+0000 7f7e41267700 1 -- 172.21.15.188:0/2945980186 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38011830 con 0x7f7e3c155df0 2023-11-04T05:27:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.534+0000 7f7e41267700 1 -- 172.21.15.188:0/2945980186 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38011990 con 0x7f7e3c155df0 2023-11-04T05:27:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.534+0000 7f7e41267700 1 -- 172.21.15.188:0/2945980186 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38011d20 con 0x7f7e3c155df0 2023-11-04T05:27:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.534+0000 7f7e444cd700 1 -- 172.21.15.188:0/2945980186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c155df0 msgr2=0x7f7e3c0cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.534+0000 7f7e444cd700 1 --2- 172.21.15.188:0/2945980186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0cc100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c09e710 tx=0x7f7e38009b20 comp rx=0 tx=0).stop 2023-11-04T05:27:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.535+0000 7f7e444cd700 1 -- 172.21.15.188:0/2945980186 shutdown_connections 2023-11-04T05:27:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.535+0000 7f7e444cd700 1 --2- 172.21.15.188:0/2945980186 >> v2:172.21.15.3:3300/0 conn(0x7f7e3c15c2e0 0x7f7e3c06ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.535+0000 7f7e444cd700 1 --2- 172.21.15.188:0/2945980186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0cc100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.535+0000 7f7e444cd700 1 -- 172.21.15.188:0/2945980186 >> 172.21.15.188:0/2945980186 conn(0x7f7e3c072560 msgr2=0x7f7e3c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.535+0000 7f7e444cd700 1 -- 172.21.15.188:0/2945980186 shutdown_connections 2023-11-04T05:27:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.535+0000 7f7e444cd700 1 -- 172.21.15.188:0/2945980186 wait complete. 2023-11-04T05:27:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.536+0000 7f7e444cd700 1 Processor -- start 2023-11-04T05:27:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.536+0000 7f7e444cd700 1 -- start start 2023-11-04T05:27:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e444cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e444cd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 0x7f7e3c0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e444cd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 0x7f7e3c0c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e444cd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c8940 con 0x7f7e3c155df0 2023-11-04T05:27:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e444cd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c8c20 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e444cd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c8f00 con 0x7f7e3c15be50 2023-11-04T05:27:59.546 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e41a68700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 0x7f7e3c0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.546 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e42a6a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 0x7f7e3c0c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.546 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e42a6a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 0x7f7e3c0c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47188/0 (socket says 172.21.15.188:47188) 2023-11-04T05:27:59.546 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e42a6a700 1 -- 172.21.15.188:0/1192819902 learned_addr learned my addr 172.21.15.188:0/1192819902 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:59.546 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e41a68700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 0x7f7e3c0c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36724/0 (socket says 172.21.15.188:36724) 2023-11-04T05:27:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.537+0000 7f7e42269700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e42a6a700 1 -- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 msgr2=0x7f7e3c0c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e42a6a700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 0x7f7e3c0c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e42a6a700 1 -- 172.21.15.188:0/1192819902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e3c155df0 msgr2=0x7f7e3c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e42a6a700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e42a6a700 1 -- 172.21.15.188:0/1192819902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380094a0 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e41a68700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 0x7f7e3c0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:27:59.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e42a6a700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 0x7f7e3c0c46c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e34007e60 tx=0x7f7e3400e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400ed30 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e34014420 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.549 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c0c91e0 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.549 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e340147d0 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.549 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c1f6ab0 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.549 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7e34014c30 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.549 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.538+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f7e3401ebe0 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.539+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c0c9700 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.541+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7e3c0c9700 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.550 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:59.548+0000 7fafbdd93200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-04T05:27:59.626 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.625+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f8a58fb6-14ea-4604-9d75-0d1b237cad96", "id": 3} v 0) v1 -- 0x7f7e3c06b960 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.632 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.630+0000 7f7e32ffd700 1 -- 172.21.15.188:0/1192819902 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8a58fb6-14ea-4604-9d75-0d1b237cad96", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e3401e460 con 0x7f7e3c0c9f20 2023-11-04T05:27:59.632 INFO:teuthology.orchestra.run.smithi188.stdout:3 2023-11-04T05:27:59.633 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 msgr2=0x7f7e3c0c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.634 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 0x7f7e3c0c46c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e34007e60 tx=0x7f7e3400e3c0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.634 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 shutdown_connections 2023-11-04T05:27:59.634 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e3c0c9f20 0x7f7e3c0c46c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.634 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e3c155df0 0x7f7e3c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.635 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 --2- 172.21.15.188:0/1192819902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e3c15be50 0x7f7e3c0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.635 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.631+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 >> 172.21.15.188:0/1192819902 conn(0x7f7e3c072560 msgr2=0x7f7e3c0b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.635 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.632+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 shutdown_connections 2023-11-04T05:27:59.635 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.632+0000 7f7e444cd700 1 -- 172.21.15.188:0/1192819902 wait complete. 2023-11-04T05:27:59.636 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:27:59.632+0000 7fafbdd93200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-04T05:27:59.645 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f8eeca37-9e36-4201-8c72-e9133f3b85a7 4 2023-11-04T05:27:59.730 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:27:59.729+0000 7f866aa6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-04T05:27:59.767 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.765+0000 7fdc48fd1700 1 Processor -- start 2023-11-04T05:27:59.767 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.765+0000 7fdc48fd1700 1 -- start start 2023-11-04T05:27:59.770 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc48fd1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc440ceaa0 0x7fdc440cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.770 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc48fd1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc4415b430 0x7fdc440caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.770 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc48fd1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4415b870 con 0x7fdc440cee80 2023-11-04T05:27:59.770 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc48fd1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc440ccc70 con 0x7fdc440ceaa0 2023-11-04T05:27:59.770 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc48fd1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc440cdee0 con 0x7fdc4415b430 2023-11-04T05:27:59.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc42d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc4415b430 0x7fdc440caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc42d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc4415b430 0x7fdc440caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47192/0 (socket says 172.21.15.188:47192) 2023-11-04T05:27:59.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc42d9d700 1 -- 172.21.15.188:0/172038927 learned_addr learned my addr 172.21.15.188:0/172038927 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:59.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.766+0000 7fdc4259c700 1 --2- 172.21.15.188:0/172038927 >> v2:172.21.15.3:3300/0 conn(0x7fdc440ceaa0 0x7fdc440cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc42d9d700 1 -- 172.21.15.188:0/172038927 >> v2:172.21.15.3:3300/0 conn(0x7fdc440ceaa0 msgr2=0x7fdc440cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:27:59.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc42d9d700 1 --2- 172.21.15.188:0/172038927 >> v2:172.21.15.3:3300/0 conn(0x7fdc440ceaa0 0x7fdc440cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc42d9d700 1 -- 172.21.15.188:0/172038927 >> v1:172.21.15.3:6789/0 conn(0x7fdc440cee80 legacy=0x7fdc4415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc42d9d700 1 -- 172.21.15.188:0/172038927 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4415baa0 con 0x7fdc4415b430 2023-11-04T05:27:59.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc42d9d700 1 --2- 172.21.15.188:0/172038927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc440caff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cac20 tx=0x7fdc3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ae7fda3bdf8391 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc4159a700 1 -- 172.21.15.188:0/172038927 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00e070 con 0x7fdc4415b430 2023-11-04T05:27:59.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc4159a700 1 -- 172.21.15.188:0/172038927 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00b910 con 0x7fdc4415b430 2023-11-04T05:27:59.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.767+0000 7fdc4159a700 1 -- 172.21.15.188:0/172038927 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00bcc0 con 0x7fdc4415b430 2023-11-04T05:27:59.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 -- 172.21.15.188:0/172038927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 msgr2=0x7fdc440caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/172038927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc440caff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cac20 tx=0x7fdc3c00b330 comp rx=0 tx=0).stop 2023-11-04T05:27:59.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 -- 172.21.15.188:0/172038927 shutdown_connections 2023-11-04T05:27:59.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/172038927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc440caff0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/172038927 >> v2:172.21.15.3:3300/0 conn(0x7fdc440ceaa0 0x7fdc440cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 -- 172.21.15.188:0/172038927 >> 172.21.15.188:0/172038927 conn(0x7fdc44072560 msgr2=0x7fdc44099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 -- 172.21.15.188:0/172038927 shutdown_connections 2023-11-04T05:27:59.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.768+0000 7fdc48fd1700 1 -- 172.21.15.188:0/172038927 wait complete. 2023-11-04T05:27:59.777 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.769+0000 7fdc48fd1700 1 Processor -- start 2023-11-04T05:27:59.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.769+0000 7fdc48fd1700 1 -- start start 2023-11-04T05:27:59.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc48fd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 0x7fdc440ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc48fd1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc440cee80 0x7fdc440b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc48fd1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc44151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:27:59.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc48fd1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440c8770 con 0x7fdc440cee80 2023-11-04T05:27:59.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc48fd1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440b38c0 con 0x7fdc4415b430 2023-11-04T05:27:59.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc48fd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440b3ba0 con 0x7fdc440ceaa0 2023-11-04T05:27:59.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc4259c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 0x7fdc440ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc42d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc44151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc4259c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 0x7fdc440ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36732/0 (socket says 172.21.15.188:36732) 2023-11-04T05:27:59.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc42d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc44151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47198/0 (socket says 172.21.15.188:47198) 2023-11-04T05:27:59.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc4259c700 1 -- 172.21.15.188:0/92456931 learned_addr learned my addr 172.21.15.188:0/92456931 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:27:59.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.770+0000 7fdc41d9b700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc440cee80 0x7fdc440b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:27:59.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc42d9d700 1 -- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 msgr2=0x7fdc440ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc42d9d700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 0x7fdc440ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc42d9d700 1 -- 172.21.15.188:0/92456931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc440cee80 msgr2=0x7fdc440b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc42d9d700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc440cee80 0x7fdc440b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc42d9d700 1 -- 172.21.15.188:0/92456931 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009b30 con 0x7fdc4415b430 2023-11-04T05:27:59.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc4259c700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 0x7fdc440ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:27:59.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc42d9d700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc44151440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c015000 tx=0x7fdc3c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:27:59.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c004880 con 0x7fdc4415b430 2023-11-04T05:27:59.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c0092e0 con 0x7fdc4415b430 2023-11-04T05:27:59.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c01c660 con 0x7fdc4415b430 2023-11-04T05:27:59.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc440b3e80 con 0x7fdc4415b430 2023-11-04T05:27:59.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc440b06d0 con 0x7fdc4415b430 2023-11-04T05:27:59.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdc3c01cac0 con 0x7fdc4415b430 2023-11-04T05:27:59.783 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.771+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fdc3c025cd0 con 0x7fdc4415b430 2023-11-04T05:27:59.783 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.772+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc440b3ce0 con 0x7fdc4415b430 2023-11-04T05:27:59.783 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.775+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdc3c043ce0 con 0x7fdc4415b430 2023-11-04T05:27:59.862 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.860+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f8eeca37-9e36-4201-8c72-e9133f3b85a7", "id": 4} v 0) v1 -- 0x7fdc440c88e0 con 0x7fdc4415b430 2023-11-04T05:27:59.867 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.866+0000 7fdc32ffd700 1 -- 172.21.15.188:0/92456931 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8eeca37-9e36-4201-8c72-e9133f3b85a7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc3c032030 con 0x7fdc4415b430 2023-11-04T05:27:59.868 INFO:teuthology.orchestra.run.smithi188.stdout:4 2023-11-04T05:27:59.869 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 msgr2=0x7fdc44151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:27:59.869 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc44151440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c015000 tx=0x7fdc3c004ad0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.869 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 shutdown_connections 2023-11-04T05:27:59.869 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc4415b430 0x7fdc44151440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.870 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc440ceaa0 0x7fdc440ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.870 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 --2- 172.21.15.188:0/92456931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc440cee80 0x7fdc440b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:27:59.870 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 >> 172.21.15.188:0/92456931 conn(0x7fdc44072560 msgr2=0x7fdc4406ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:27:59.870 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 shutdown_connections 2023-11-04T05:27:59.870 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.867+0000 7fdc48fd1700 1 -- 172.21.15.188:0/92456931 wait complete. 2023-11-04T05:27:59.881 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0 5 2023-11-04T05:28:00.001 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.999+0000 7fb86513c700 1 Processor -- start 2023-11-04T05:28:00.001 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.999+0000 7fb86513c700 1 -- start start 2023-11-04T05:28:00.003 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.999+0000 7fb86513c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb86015dd80 0x7fb86006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.004 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:27:59.999+0000 7fb86513c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 0x7fb860161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.004 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb86513c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86006d920 con 0x7fb8600b3110 2023-11-04T05:28:00.004 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb86513c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb860162250 con 0x7fb86015dd80 2023-11-04T05:28:00.004 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb86513c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86006bee0 con 0x7fb8601616e0 2023-11-04T05:28:00.004 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85e59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 0x7fb860161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.005 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb86015dd80 0x7fb86006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.005 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb86015dd80 0x7fb86006c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47200/0 (socket says 172.21.15.188:47200) 2023-11-04T05:28:00.005 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85e59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 0x7fb860161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36740/0 (socket says 172.21.15.188:36740) 2023-11-04T05:28:00.005 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2825948793 learned_addr learned my addr 172.21.15.188:0/2825948793 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:28:00.005 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 msgr2=0x7fb860161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:00.006 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 --2- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 0x7fb860161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.006 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2825948793 >> v1:172.21.15.188:6789/0 conn(0x7fb8600b3110 legacy=0x7fb8600cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.006 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2825948793 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86006c1c0 con 0x7fb86015dd80 2023-11-04T05:28:00.006 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85dd9b700 1 -- 172.21.15.188:0/2825948793 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016842573 0 0) 0x7fb86006d920 con 0x7fb8600b3110 2023-11-04T05:28:00.006 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85e59c700 1 --2- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 0x7fb860161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:00.007 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85ed9d700 1 --2- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86015dd80 0x7fb86006c9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8500098f0 tx=0x7fb850009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3788f5c7235cde6f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.007 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85dd9b700 1 -- 172.21.15.188:0/2825948793 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850011830 con 0x7fb86015dd80 2023-11-04T05:28:00.007 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85dd9b700 1 -- 172.21.15.188:0/2825948793 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850011990 con 0x7fb86015dd80 2023-11-04T05:28:00.007 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.000+0000 7fb85dd9b700 1 -- 172.21.15.188:0/2825948793 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850011d20 con 0x7fb86015dd80 2023-11-04T05:28:00.007 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 -- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86015dd80 msgr2=0x7fb86006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.008 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 --2- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86015dd80 0x7fb86006c9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8500098f0 tx=0x7fb850009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:00.008 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 -- 172.21.15.188:0/2825948793 shutdown_connections 2023-11-04T05:28:00.008 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 --2- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86015dd80 0x7fb86006c9c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.008 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 --2- 172.21.15.188:0/2825948793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8601616e0 0x7fb860161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.008 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 -- 172.21.15.188:0/2825948793 >> 172.21.15.188:0/2825948793 conn(0x7fb8600ce110 msgr2=0x7fb8600ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.009 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 -- 172.21.15.188:0/2825948793 shutdown_connections 2023-11-04T05:28:00.009 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.001+0000 7fb86513c700 1 -- 172.21.15.188:0/2825948793 wait complete. 2023-11-04T05:28:00.011 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.002+0000 7fb86513c700 1 Processor -- start 2023-11-04T05:28:00.012 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 -- start start 2023-11-04T05:28:00.012 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8600b3110 0x7fb8600b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.012 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 0x7fb8600d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.012 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb860158270 0x7fb860156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.012 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600b50f0 con 0x7fb860158270 2023-11-04T05:28:00.013 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600cb1f0 con 0x7fb8601616e0 2023-11-04T05:28:00.013 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb86513c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600ca620 con 0x7fb8600b3110 2023-11-04T05:28:00.013 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb85e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 0x7fb8600d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.013 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb85ed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8600b3110 0x7fb8600b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.013 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb85e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 0x7fb8600d18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47208/0 (socket says 172.21.15.188:47208) 2023-11-04T05:28:00.014 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.003+0000 7fb85e59c700 1 -- 172.21.15.188:0/2092968262 learned_addr learned my addr 172.21.15.188:0/2092968262 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:28:00.014 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85f59e700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb860158270 0x7fb860156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.014 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 msgr2=0x7fb8600d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.014 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85ed9d700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 0x7fb8600d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2092968262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb860158270 msgr2=0x7fb860156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85ed9d700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb860158270 0x7fb860156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85ed9d700 1 -- 172.21.15.188:0/2092968262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8540094a0 con 0x7fb8600b3110 2023-11-04T05:28:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85f59e700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb860158270 0x7fb860156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85e59c700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 0x7fb8600d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb85ed9d700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8600b3110 0x7fb8600b6d90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb850016000 tx=0x7fb850004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850004400 con 0x7fb8600b3110 2023-11-04T05:28:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85001f070 con 0x7fb8600b3110 2023-11-04T05:28:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500038c0 con 0x7fb8600b3110 2023-11-04T05:28:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb850009550 con 0x7fb8600b3110 2023-11-04T05:28:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8600caba0 con 0x7fb8600b3110 2023-11-04T05:28:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb850021030 con 0x7fb8600b3110 2023-11-04T05:28:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.004+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb850024430 con 0x7fb8600b3110 2023-11-04T05:28:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.005+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb860074df0 con 0x7fb8600b3110 2023-11-04T05:28:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.007+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb850015020 con 0x7fb8600b3110 2023-11-04T05:28:00.090 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:00.089+0000 7fafbdd93200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-04T05:28:00.094 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.091+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0", "id": 5} v 0) v1 -- 0x7fb8601591a0 con 0x7fb8600b3110 2023-11-04T05:28:00.101 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.099+0000 7fb8477fe700 1 -- 172.21.15.188:0/2092968262 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb85001a450 con 0x7fb8600b3110 2023-11-04T05:28:00.101 INFO:teuthology.orchestra.run.smithi188.stdout:5 2023-11-04T05:28:00.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8600b3110 msgr2=0x7fb8600b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8600b3110 0x7fb8600b6d90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb850016000 tx=0x7fb850004750 comp rx=0 tx=0).stop 2023-11-04T05:28:00.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 shutdown_connections 2023-11-04T05:28:00.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb860158270 0x7fb860156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8600b3110 0x7fb8600b6d90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 --2- 172.21.15.188:0/2092968262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8601616e0 0x7fb8600d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 >> 172.21.15.188:0/2092968262 conn(0x7fb8600ce110 msgr2=0x7fb8600cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.104 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 shutdown_connections 2023-11-04T05:28:00.104 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.100+0000 7fb86513c700 1 -- 172.21.15.188:0/2092968262 wait complete. 2023-11-04T05:28:00.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f23194fe-1570-4fa1-af76-fe10e798e7bf 6 2023-11-04T05:28:00.258 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.256+0000 7f87542f5700 1 Processor -- start 2023-11-04T05:28:00.258 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.256+0000 7f87542f5700 1 -- start start 2023-11-04T05:28:00.261 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f87542f5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.261 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f87542f5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f874c0d11f0 0x7f874c15f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.261 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f87542f5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c070a90 con 0x7f874c0d0d70 2023-11-04T05:28:00.261 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f87542f5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0718f0 con 0x7f874c0d11f0 2023-11-04T05:28:00.262 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f87542f5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0c5720 con 0x7f874c15ba10 2023-11-04T05:28:00.262 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f8751890700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f874c0d0d70 0x7f874c0b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38288/0 (socket says 172.21.15.188:38288) 2023-11-04T05:28:00.262 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f8751890700 1 -- 172.21.15.188:0/1225665818 learned_addr learned my addr 172.21.15.188:0/1225665818 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-04T05:28:00.262 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f8752091700 1 --2- 172.21.15.188:0/1225665818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.262 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f8752892700 1 --2- 172.21.15.188:0/1225665818 >> v2:172.21.15.3:3300/0 conn(0x7f874c0d11f0 0x7f874c15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.257+0000 7f875108f700 1 -- 172.21.15.188:0/1225665818 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970346480 0 0) 0x7f874c070a90 con 0x7f874c0d0d70 2023-11-04T05:28:00.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f875108f700 1 -- 172.21.15.188:0/1225665818 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87380035b0 con 0x7f874c0d0d70 2023-11-04T05:28:00.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f8752091700 1 -- 172.21.15.188:0/1225665818 >> v2:172.21.15.3:3300/0 conn(0x7f874c0d11f0 msgr2=0x7f874c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:00.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f8752091700 1 --2- 172.21.15.188:0/1225665818 >> v2:172.21.15.3:3300/0 conn(0x7f874c0d11f0 0x7f874c15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.263 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f8752091700 1 -- 172.21.15.188:0/1225665818 >> v1:172.21.15.188:6789/0 conn(0x7f874c0d0d70 legacy=0x7f874c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f8752091700 1 -- 172.21.15.188:0/1225665818 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c070ce0 con 0x7f874c15ba10 2023-11-04T05:28:00.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f8752091700 1 --2- 172.21.15.188:0/1225665818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0c51e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8748009840 tx=0x7f8748009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ff8c940cd1c8ba server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f875108f700 1 -- 172.21.15.188:0/1225665818 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748011830 con 0x7f874c15ba10 2023-11-04T05:28:00.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f875108f700 1 -- 172.21.15.188:0/1225665818 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748011990 con 0x7f874c15ba10 2023-11-04T05:28:00.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f875108f700 1 -- 172.21.15.188:0/1225665818 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748011d20 con 0x7f874c15ba10 2023-11-04T05:28:00.264 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f87542f5700 1 -- 172.21.15.188:0/1225665818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 msgr2=0x7f874c0c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.258+0000 7f87542f5700 1 --2- 172.21.15.188:0/1225665818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0c51e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8748009840 tx=0x7f8748009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:00.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.259+0000 7f87542f5700 1 -- 172.21.15.188:0/1225665818 shutdown_connections 2023-11-04T05:28:00.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.259+0000 7f87542f5700 1 --2- 172.21.15.188:0/1225665818 >> v2:172.21.15.3:3300/0 conn(0x7f874c0d11f0 0x7f874c15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.259+0000 7f87542f5700 1 --2- 172.21.15.188:0/1225665818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0c51e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.265 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.259+0000 7f87542f5700 1 -- 172.21.15.188:0/1225665818 >> 172.21.15.188:0/1225665818 conn(0x7f874c0ce110 msgr2=0x7f874c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.259+0000 7f87542f5700 1 -- 172.21.15.188:0/1225665818 shutdown_connections 2023-11-04T05:28:00.266 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.259+0000 7f87542f5700 1 -- 172.21.15.188:0/1225665818 wait complete. 2023-11-04T05:28:00.268 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.260+0000 7f87542f5700 1 Processor -- start 2023-11-04T05:28:00.268 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.260+0000 7f87542f5700 1 -- start start 2023-11-04T05:28:00.269 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.260+0000 7f87542f5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874c0d0d70 0x7f874c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.269 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.260+0000 7f87542f5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.269 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f87542f5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0cb760 0x7f874c06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.269 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f87542f5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c15ee90 con 0x7f874c0d0d70 2023-11-04T05:28:00.269 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f87542f5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c06bcd0 con 0x7f874c0cb760 2023-11-04T05:28:00.269 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f87542f5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c06bfb0 con 0x7f874c15ba10 2023-11-04T05:28:00.270 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.270 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8752892700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0cb760 0x7f874c06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.270 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:36758/0 (socket says 172.21.15.188:36758) 2023-11-04T05:28:00.270 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 -- 172.21.15.188:0/1578087905 learned_addr learned my addr 172.21.15.188:0/1578087905 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:28:00.270 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8752091700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874c0d0d70 0x7f874c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.271 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 -- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0cb760 msgr2=0x7f874c06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.271 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0cb760 0x7f874c06cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.271 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 -- 172.21.15.188:0/1578087905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874c0d0d70 msgr2=0x7f874c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.271 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874c0d0d70 0x7f874c15e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.271 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 -- 172.21.15.188:0/1578087905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480094a0 con 0x7f874c15ba10 2023-11-04T05:28:00.272 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8752892700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0cb760 0x7f874c06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:00.272 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8751890700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0cb220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f873c00c700 tx=0x7f873c00ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.272 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c005d70 con 0x7f874c15ba10 2023-11-04T05:28:00.272 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c0070d0 con 0x7f874c15ba10 2023-11-04T05:28:00.272 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c06c290 con 0x7f874c15ba10 2023-11-04T05:28:00.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c007460 con 0x7f874c15ba10 2023-11-04T05:28:00.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c06c850 con 0x7f874c15ba10 2023-11-04T05:28:00.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.261+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f873c01c440 con 0x7f874c15ba10 2023-11-04T05:28:00.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.262+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f873c0129f0 con 0x7f874c15ba10 2023-11-04T05:28:00.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.262+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c074df0 con 0x7f874c15ba10 2023-11-04T05:28:00.273 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.264+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f873c017070 con 0x7f874c15ba10 2023-11-04T05:28:00.351 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.349+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f23194fe-1570-4fa1-af76-fe10e798e7bf", "id": 6} v 0) v1 -- 0x7f874c0cee70 con 0x7f874c15ba10 2023-11-04T05:28:00.359 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.357+0000 7f8742ffd700 1 -- 172.21.15.188:0/1578087905 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f23194fe-1570-4fa1-af76-fe10e798e7bf", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f873c0124d0 con 0x7f874c15ba10 2023-11-04T05:28:00.359 INFO:teuthology.orchestra.run.smithi188.stdout:6 2023-11-04T05:28:00.360 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 msgr2=0x7f874c0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0cb220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f873c00c700 tx=0x7f873c00ca10 comp rx=0 tx=0).stop 2023-11-04T05:28:00.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 shutdown_connections 2023-11-04T05:28:00.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0cb760 0x7f874c06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f874c0d0d70 0x7f874c15e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.361 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 --2- 172.21.15.188:0/1578087905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c15ba10 0x7f874c0cb220 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.362 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.358+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 >> 172.21.15.188:0/1578087905 conn(0x7f874c0ce110 msgr2=0x7f874c0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.362 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.359+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 shutdown_connections 2023-11-04T05:28:00.362 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.359+0000 7f87542f5700 1 -- 172.21.15.188:0/1578087905 wait complete. 2023-11-04T05:28:00.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new d81eabe1-34b1-43d7-b5c5-9ac505310564 7 2023-11-04T05:28:00.408 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:00.406+0000 7f866aa6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-04T05:28:00.480 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:00.479+0000 7f866aa6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-04T05:28:00.496 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.494+0000 7faf41c57700 1 Processor -- start 2023-11-04T05:28:00.496 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.494+0000 7faf41c57700 1 -- start start 2023-11-04T05:28:00.507 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf41c57700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf3c15d9a0 0x7faf3c0d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.508 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf41c57700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faf3c0b2620 0x7faf3c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.508 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf41c57700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0ceed0 con 0x7faf3c0b0500 2023-11-04T05:28:00.508 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf41c57700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0d1520 con 0x7faf3c0b2620 2023-11-04T05:28:00.509 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf41c57700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0cf120 con 0x7faf3c15d9a0 2023-11-04T05:28:00.509 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3b7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf3c15d9a0 0x7faf3c0d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.509 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3bfff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7faf3c0b0500 0x7faf3c161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:38304/0 (socket says 172.21.15.188:38304) 2023-11-04T05:28:00.510 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3b7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf3c15d9a0 0x7faf3c0d3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47224/0 (socket says 172.21.15.188:47224) 2023-11-04T05:28:00.510 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3bfff700 1 -- 172.21.15.188:0/2307917068 learned_addr learned my addr 172.21.15.188:0/2307917068 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-04T05:28:00.510 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3affd700 1 --2- 172.21.15.188:0/2307917068 >> v2:172.21.15.3:3300/0 conn(0x7faf3c0b2620 0x7faf3c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.511 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3b7fe700 1 -- 172.21.15.188:0/2307917068 >> v2:172.21.15.3:3300/0 conn(0x7faf3c0b2620 msgr2=0x7faf3c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:00.511 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3b7fe700 1 --2- 172.21.15.188:0/2307917068 >> v2:172.21.15.3:3300/0 conn(0x7faf3c0b2620 0x7faf3c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.511 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3b7fe700 1 -- 172.21.15.188:0/2307917068 >> v1:172.21.15.188:6789/0 conn(0x7faf3c0b0500 legacy=0x7faf3c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.512 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3b7fe700 1 -- 172.21.15.188:0/2307917068 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0cf400 con 0x7faf3c15d9a0 2023-11-04T05:28:00.512 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.495+0000 7faf3a7fc700 1 -- 172.21.15.188:0/2307917068 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295450925 0 0) 0x7faf3c0ceed0 con 0x7faf3c0b0500 2023-11-04T05:28:00.512 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf3b7fe700 1 --2- 172.21.15.188:0/2307917068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c0d3fc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0b64a0 tx=0x7faf2c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7dd4ffd45f44fd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.513 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf3a7fc700 1 -- 172.21.15.188:0/2307917068 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c011830 con 0x7faf3c15d9a0 2023-11-04T05:28:00.513 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf3a7fc700 1 -- 172.21.15.188:0/2307917068 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c011990 con 0x7faf3c15d9a0 2023-11-04T05:28:00.514 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf3a7fc700 1 -- 172.21.15.188:0/2307917068 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c011d20 con 0x7faf3c15d9a0 2023-11-04T05:28:00.514 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 -- 172.21.15.188:0/2307917068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 msgr2=0x7faf3c0d3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.514 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 --2- 172.21.15.188:0/2307917068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c0d3fc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0b64a0 tx=0x7faf2c009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:00.515 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 -- 172.21.15.188:0/2307917068 shutdown_connections 2023-11-04T05:28:00.515 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 --2- 172.21.15.188:0/2307917068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c0d3fc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.515 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 --2- 172.21.15.188:0/2307917068 >> v2:172.21.15.3:3300/0 conn(0x7faf3c0b2620 0x7faf3c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.516 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 -- 172.21.15.188:0/2307917068 >> 172.21.15.188:0/2307917068 conn(0x7faf3c08aff0 msgr2=0x7faf3c0d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.516 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.496+0000 7faf41c57700 1 -- 172.21.15.188:0/2307917068 shutdown_connections 2023-11-04T05:28:00.516 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.497+0000 7faf41c57700 1 -- 172.21.15.188:0/2307917068 wait complete. 2023-11-04T05:28:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.498+0000 7faf41c57700 1 Processor -- start 2023-11-04T05:28:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.498+0000 7faf41c57700 1 -- start start 2023-11-04T05:28:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf41c57700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0b0500 0x7faf3c14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf41c57700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3c0b2620 0x7faf3c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf41c57700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c1f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.519 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf41c57700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c159d20 con 0x7faf3c0b0500 2023-11-04T05:28:00.519 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf41c57700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c15a2e0 con 0x7faf3c15d9a0 2023-11-04T05:28:00.519 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf41c57700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c157840 con 0x7faf3c0b2620 2023-11-04T05:28:00.519 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf3bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c1f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.520 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf3affd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3c0b2620 0x7faf3c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.520 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf3bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c1f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47234/0 (socket says 172.21.15.188:47234) 2023-11-04T05:28:00.520 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf3bfff700 1 -- 172.21.15.188:0/640272415 learned_addr learned my addr 172.21.15.188:0/640272415 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-04T05:28:00.521 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.499+0000 7faf3b7fe700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0b0500 0x7faf3c14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.521 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3affd700 1 -- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 msgr2=0x7faf3c1f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.521 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3affd700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c1f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3affd700 1 -- 172.21.15.188:0/640272415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0b0500 msgr2=0x7faf3c14c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3affd700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0b0500 0x7faf3c14c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3affd700 1 -- 172.21.15.188:0/640272415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0094a0 con 0x7faf3c0b2620 2023-11-04T05:28:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3bfff700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c1f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf3affd700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3c0b2620 0x7faf3c1f6ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf3000e7e0 tx=0x7faf3000eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000c070 con 0x7faf3c0b2620 2023-11-04T05:28:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30013610 con 0x7faf3c0b2620 2023-11-04T05:28:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300139a0 con 0x7faf3c0b2620 2023-11-04T05:28:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c157b20 con 0x7faf3c0b2620 2023-11-04T05:28:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.500+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c157530 con 0x7faf3c0b2620 2023-11-04T05:28:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.501+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faf3001c400 con 0x7faf3c0b2620 2023-11-04T05:28:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.501+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c15a450 con 0x7faf3c0b2620 2023-11-04T05:28:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.501+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7faf3001ba10 con 0x7faf3c0b2620 2023-11-04T05:28:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.503+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7faf30017020 con 0x7faf3c0b2620 2023-11-04T05:28:00.589 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.587+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d81eabe1-34b1-43d7-b5c5-9ac505310564", "id": 7} v 0) v1 -- 0x7faf3c0d0e60 con 0x7faf3c0b2620 2023-11-04T05:28:00.597 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.595+0000 7faf23fff700 1 -- 172.21.15.188:0/640272415 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d81eabe1-34b1-43d7-b5c5-9ac505310564", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf3000f040 con 0x7faf3c0b2620 2023-11-04T05:28:00.597 INFO:teuthology.orchestra.run.smithi188.stdout:7 2023-11-04T05:28:00.599 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3c0b2620 msgr2=0x7faf3c1f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.599 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3c0b2620 0x7faf3c1f6ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf3000e7e0 tx=0x7faf3000eaf0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.599 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 shutdown_connections 2023-11-04T05:28:00.599 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3c15d9a0 0x7faf3c1f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf3c0b0500 0x7faf3c14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 --2- 172.21.15.188:0/640272415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3c0b2620 0x7faf3c1f6ae0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 >> 172.21.15.188:0/640272415 conn(0x7faf3c08aff0 msgr2=0x7faf3c06c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 shutdown_connections 2023-11-04T05:28:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-04T05:28:00.597+0000 7faf41c57700 1 -- 172.21.15.188:0/640272415 wait complete. 2023-11-04T05:28:00.615 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-04T05:28:00.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-04T05:28:00.617 INFO:tasks.ceph.osd.0:Started 2023-11-04T05:28:00.618 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-04T05:28:00.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-04T05:28:00.620 INFO:tasks.ceph.osd.1:Started 2023-11-04T05:28:00.620 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-04T05:28:00.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-04T05:28:00.623 INFO:tasks.ceph.osd.2:Started 2023-11-04T05:28:00.623 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-04T05:28:00.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-04T05:28:00.626 INFO:tasks.ceph.osd.3:Started 2023-11-04T05:28:00.626 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-04T05:28:00.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-04T05:28:00.656 INFO:tasks.ceph.osd.4:Started 2023-11-04T05:28:00.657 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-04T05:28:00.657 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-04T05:28:00.659 INFO:tasks.ceph.osd.5:Started 2023-11-04T05:28:00.659 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-04T05:28:00.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-04T05:28:00.662 INFO:tasks.ceph.osd.6:Started 2023-11-04T05:28:00.662 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-04T05:28:00.663 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-04T05:28:00.665 INFO:tasks.ceph.osd.7:Started 2023-11-04T05:28:00.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:00.791 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:00.789+0000 7fafbdd93200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-04T05:28:00.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.811+0000 7f4adc181700 1 Processor -- start 2023-11-04T05:28:00.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.812+0000 7f4adc181700 1 -- start start 2023-11-04T05:28:00.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.812+0000 7f4adc181700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ad406ab20 0x7f4ad40728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.812+0000 7f4adc181700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad40cc7a0 0x7f4ad40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.812+0000 7f4adc181700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad4072df0 con 0x7f4ad406af00 2023-11-04T05:28:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4adc181700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad40cf050 con 0x7f4ad406ab20 2023-11-04T05:28:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4adc181700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad40cd780 con 0x7f4ad40cc7a0 2023-11-04T05:28:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ada71e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4ad406af00 0x7f4ad415ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56368/0 (socket says 172.21.15.3:56368) 2023-11-04T05:28:00.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ada71e700 1 -- 172.21.15.3:0/3073219254 learned_addr learned my addr 172.21.15.3:0/3073219254 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad9f1d700 1 --2- 172.21.15.3:0/3073219254 >> v2:172.21.15.188:3300/0 conn(0x7f4ad406ab20 0x7f4ad40728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad971c700 1 --2- 172.21.15.3:0/3073219254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad40cc7a0 0x7f4ad40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad8f1b700 1 -- 172.21.15.3:0/3073219254 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286026551 0 0) 0x7f4ad4072df0 con 0x7f4ad406af00 2023-11-04T05:28:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad8f1b700 1 -- 172.21.15.3:0/3073219254 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac00035b0 con 0x7f4ad406af00 2023-11-04T05:28:00.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad971c700 1 -- 172.21.15.3:0/3073219254 >> v2:172.21.15.188:3300/0 conn(0x7f4ad406ab20 msgr2=0x7f4ad40728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:00.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad971c700 1 --2- 172.21.15.3:0/3073219254 >> v2:172.21.15.188:3300/0 conn(0x7f4ad406ab20 0x7f4ad40728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad971c700 1 -- 172.21.15.3:0/3073219254 >> v1:172.21.15.3:6789/0 conn(0x7f4ad406af00 legacy=0x7f4ad415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad971c700 1 -- 172.21.15.3:0/3073219254 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad40cda00 con 0x7f4ad40cc7a0 2023-11-04T05:28:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad9f1d700 1 --2- 172.21.15.3:0/3073219254 >> v2:172.21.15.188:3300/0 conn(0x7f4ad406ab20 0x7f4ad40728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad971c700 1 --2- 172.21.15.3:0/3073219254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad40cc7a0 0x7f4ad40cd240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009820 tx=0x7f4ad0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332d655805fc0b3b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.813+0000 7f4ad8f1b700 1 -- 172.21.15.3:0/3073219254 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0005e30 con 0x7f4ad40cc7a0 2023-11-04T05:28:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4ad8f1b700 1 -- 172.21.15.3:0/3073219254 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad0006030 con 0x7f4ad40cc7a0 2023-11-04T05:28:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4ad8f1b700 1 -- 172.21.15.3:0/3073219254 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00063c0 con 0x7f4ad40cc7a0 2023-11-04T05:28:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 -- 172.21.15.3:0/3073219254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad40cc7a0 msgr2=0x7f4ad40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 --2- 172.21.15.3:0/3073219254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad40cc7a0 0x7f4ad40cd240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009820 tx=0x7f4ad0009b30 comp rx=0 tx=0).stop 2023-11-04T05:28:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 -- 172.21.15.3:0/3073219254 shutdown_connections 2023-11-04T05:28:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 --2- 172.21.15.3:0/3073219254 >> v2:172.21.15.188:3300/0 conn(0x7f4ad406ab20 0x7f4ad40728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 --2- 172.21.15.3:0/3073219254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad40cc7a0 0x7f4ad40cd240 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 -- 172.21.15.3:0/3073219254 >> 172.21.15.3:0/3073219254 conn(0x7f4ad406df90 msgr2=0x7f4ad406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 -- 172.21.15.3:0/3073219254 shutdown_connections 2023-11-04T05:28:00.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.814+0000 7f4adc181700 1 -- 172.21.15.3:0/3073219254 wait complete. 2023-11-04T05:28:00.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.815+0000 7f4adc181700 1 Processor -- start 2023-11-04T05:28:00.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.815+0000 7f4adc181700 1 -- start start 2023-11-04T05:28:00.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4adc181700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad406ab20 0x7f4ad40c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4adc181700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 0x7f4ad40af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4adc181700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad40cc7a0 0x7f4ad4143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4adc181700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad4154af0 con 0x7f4ad406af00 2023-11-04T05:28:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4adc181700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad40c6a10 con 0x7f4ad40cc7a0 2023-11-04T05:28:00.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4adc181700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad40c5c50 con 0x7f4ad406ab20 2023-11-04T05:28:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4ad971c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 0x7f4ad40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4ad971c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 0x7f4ad40af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51024/0 (socket says 172.21.15.3:51024) 2023-11-04T05:28:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4ad971c700 1 -- 172.21.15.3:0/559569438 learned_addr learned my addr 172.21.15.3:0/559569438 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4ad9f1d700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad406ab20 0x7f4ad40c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.816+0000 7f4ada71e700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad40cc7a0 0x7f4ad4143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4ad971c700 1 -- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad406ab20 msgr2=0x7f4ad40c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4ad971c700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad406ab20 0x7f4ad40c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4ad971c700 1 -- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad40cc7a0 msgr2=0x7f4ad4143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4ad971c700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad40cc7a0 0x7f4ad4143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4ad971c700 1 -- 172.21.15.3:0/559569438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40094a0 con 0x7f4ad406af00 2023-11-04T05:28:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4ad971c700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 0x7f4ad40af240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4072f80 tx=0x7f4ad0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00061a0 con 0x7f4ad406af00 2023-11-04T05:28:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad0020070 con 0x7f4ad406af00 2023-11-04T05:28:00.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad0009430 con 0x7f4ad406af00 2023-11-04T05:28:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0006560 con 0x7f4ad406af00 2023-11-04T05:28:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad40c6210 con 0x7f4ad406af00 2023-11-04T05:28:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4ad0022030 con 0x7f4ad406af00 2023-11-04T05:28:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.817+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4ad0003dd0 con 0x7f4ad406af00 2023-11-04T05:28:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.818+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad4074df0 con 0x7f4ad406af00 2023-11-04T05:28:00.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.820+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4ad00036a0 con 0x7f4ad406af00 2023-11-04T05:28:00.866 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:00.865+0000 7fafbdd93200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-04T05:28:00.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.892+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ad40cd020 con 0x7f4ad406af00 2023-11-04T05:28:00.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.892+0000 7f4aca7fc700 1 -- 172.21.15.3:0/559569438 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4ad0003840 con 0x7f4ad406af00 2023-11-04T05:28:00.894 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:00.894 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:00.589881+0000","last_up_change":"0.000000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 msgr2=0x7f4ad40af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 0x7f4ad40af240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4072f80 tx=0x7f4ad0004750 comp rx=0 tx=0).stop 2023-11-04T05:28:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 shutdown_connections 2023-11-04T05:28:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad40cc7a0 0x7f4ad4143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad406ab20 0x7f4ad40c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 --2- 172.21.15.3:0/559569438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ad406af00 0x7f4ad40af240 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:00.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 >> 172.21.15.3:0/559569438 conn(0x7f4ad406df90 msgr2=0x7f4ad415aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:00.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 shutdown_connections 2023-11-04T05:28:00.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:00.893+0000 7f4adc181700 1 -- 172.21.15.3:0/559569438 wait complete. 2023-11-04T05:28:00.906 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-04T05:28:00.907 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-04T05:28:01.226 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:01.224+0000 7f866aa6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-04T05:28:01.256 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T05:28:01.255+0000 7f7dc5152640 -1 Falling back to public interface 2023-11-04T05:28:01.257 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T05:28:01.255+0000 7f1f78a01640 -1 Falling back to public interface 2023-11-04T05:28:01.257 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T05:28:01.255+0000 7f7dbc510700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-11-04T05:28:01.266 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T05:28:01.265+0000 7f58b5cc8640 -1 Falling back to public interface 2023-11-04T05:28:01.267 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T05:28:01.265+0000 7f98005cd640 -1 Falling back to public interface 2023-11-04T05:28:01.277 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T05:28:01.275+0000 7eff32107640 -1 Falling back to public interface 2023-11-04T05:28:01.281 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T05:28:01.279+0000 7f8a33c55640 -1 Falling back to public interface 2023-11-04T05:28:01.282 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T05:28:01.280+0000 7f104f806640 -1 Falling back to public interface 2023-11-04T05:28:01.290 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T05:28:01.288+0000 7fdae79e3640 -1 Falling back to public interface 2023-11-04T05:28:01.309 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:01.307+0000 7f866aa6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-04T05:28:01.450 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:01.448+0000 7f866aa6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-04T05:28:01.593 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:01.592+0000 7fafbdd93200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-04T05:28:01.674 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:01.673+0000 7fafbdd93200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-04T05:28:01.684 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:01.683+0000 7f866aa6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-04T05:28:01.713 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:01.826 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:01.825+0000 7fafbdd93200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-04T05:28:01.831 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.830+0000 7f30607f8700 1 Processor -- start 2023-11-04T05:28:01.831 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.830+0000 7f30607f8700 1 -- start start 2023-11-04T05:28:01.834 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.831+0000 7f30607f8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f305815a850 0x7f30580b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:01.835 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.831+0000 7f30607f8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3058072a40 0x7f30580b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:01.835 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.831+0000 7f30607f8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30580cbf30 con 0x7f3058072ec0 2023-11-04T05:28:01.835 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.831+0000 7f30607f8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30580d03d0 con 0x7f305815a850 2023-11-04T05:28:01.835 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.831+0000 7f30607f8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30580cf850 con 0x7f3058072a40 2023-11-04T05:28:01.836 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305ed95700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3058072ec0 0x7f30580cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56380/0 (socket says 172.21.15.3:56380) 2023-11-04T05:28:01.836 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305ed95700 1 -- 172.21.15.3:0/942329567 learned_addr learned my addr 172.21.15.3:0/942329567 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:01.836 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305dd93700 1 --2- 172.21.15.3:0/942329567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3058072a40 0x7f30580b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:01.836 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305e594700 1 --2- 172.21.15.3:0/942329567 >> v2:172.21.15.188:3300/0 conn(0x7f305815a850 0x7f30580b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:01.837 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305d592700 1 -- 172.21.15.3:0/942329567 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627271431 0 0) 0x7f30580cbf30 con 0x7f3058072ec0 2023-11-04T05:28:01.837 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305d592700 1 -- 172.21.15.3:0/942329567 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30440035b0 con 0x7f3058072ec0 2023-11-04T05:28:01.837 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305dd93700 1 -- 172.21.15.3:0/942329567 >> v2:172.21.15.188:3300/0 conn(0x7f305815a850 msgr2=0x7f30580b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:01.837 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305dd93700 1 --2- 172.21.15.3:0/942329567 >> v2:172.21.15.188:3300/0 conn(0x7f305815a850 0x7f30580b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.838 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305dd93700 1 -- 172.21.15.3:0/942329567 >> v1:172.21.15.3:6789/0 conn(0x7f3058072ec0 legacy=0x7f30580cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:01.838 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305dd93700 1 -- 172.21.15.3:0/942329567 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580cfb30 con 0x7f3058072a40 2023-11-04T05:28:01.838 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305e594700 1 --2- 172.21.15.3:0/942329567 >> v2:172.21.15.188:3300/0 conn(0x7f305815a850 0x7f30580b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:01.838 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305dd93700 1 --2- 172.21.15.3:0/942329567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3058072a40 0x7f30580b2170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3048009840 tx=0x7f3048009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5979fa25498634 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:01.838 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305d592700 1 -- 172.21.15.3:0/942329567 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048011830 con 0x7f3058072a40 2023-11-04T05:28:01.839 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305d592700 1 -- 172.21.15.3:0/942329567 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048011990 con 0x7f3058072a40 2023-11-04T05:28:01.839 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.832+0000 7f305d592700 1 -- 172.21.15.3:0/942329567 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048011d00 con 0x7f3058072a40 2023-11-04T05:28:01.840 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 -- 172.21.15.3:0/942329567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3058072a40 msgr2=0x7f30580b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:01.840 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 --2- 172.21.15.3:0/942329567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3058072a40 0x7f30580b2170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3048009840 tx=0x7f3048009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:01.840 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 -- 172.21.15.3:0/942329567 shutdown_connections 2023-11-04T05:28:01.840 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 --2- 172.21.15.3:0/942329567 >> v2:172.21.15.188:3300/0 conn(0x7f305815a850 0x7f30580b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.840 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 --2- 172.21.15.3:0/942329567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3058072a40 0x7f30580b2170 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.841 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 -- 172.21.15.3:0/942329567 >> 172.21.15.3:0/942329567 conn(0x7f30580cd170 msgr2=0x7f30580cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:01.841 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 -- 172.21.15.3:0/942329567 shutdown_connections 2023-11-04T05:28:01.841 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.833+0000 7f30607f8700 1 -- 172.21.15.3:0/942329567 wait complete. 2023-11-04T05:28:01.844 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.834+0000 7f30607f8700 1 Processor -- start 2023-11-04T05:28:01.844 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.834+0000 7f30607f8700 1 -- start start 2023-11-04T05:28:01.844 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f30607f8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 0x7f305815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:01.845 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f30607f8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3058072ec0 0x7f3058152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:01.845 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f30607f8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f305815a850 0x7f30580ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:01.845 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f30607f8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30581531f0 con 0x7f3058072a40 2023-11-04T05:28:01.845 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f30607f8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30580adc90 con 0x7f3058072ec0 2023-11-04T05:28:01.845 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f30607f8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30580ace20 con 0x7f305815a850 2023-11-04T05:28:01.846 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 0x7f305815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:01.846 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 0x7f305815d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51034/0 (socket says 172.21.15.3:51034) 2023-11-04T05:28:01.846 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 -- 172.21.15.3:0/1749604858 learned_addr learned my addr 172.21.15.3:0/1749604858 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:01.846 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305dd93700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3058072ec0 0x7f3058152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:01.847 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305ed95700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f305815a850 0x7f30580ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:01.847 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 -- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f305815a850 msgr2=0x7f30580ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:01.847 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f305815a850 0x7f30580ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.847 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 -- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3058072ec0 msgr2=0x7f3058152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:01.848 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.835+0000 7f305e594700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3058072ec0 0x7f3058152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.848 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f305e594700 1 -- 172.21.15.3:0/1749604858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540094a0 con 0x7f3058072a40 2023-11-04T05:28:01.848 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f305dd93700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3058072ec0 0x7f3058152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:01.848 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f305e594700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 0x7f305815d6d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f305400ebe0 tx=0x7f305400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:01.848 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054009760 con 0x7f3058072a40 2023-11-04T05:28:01.849 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400c850 con 0x7f3058072a40 2023-11-04T05:28:01.849 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480094a0 con 0x7f3058072a40 2023-11-04T05:28:01.849 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305400cbe0 con 0x7f3058072a40 2023-11-04T05:28:01.849 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30580ad360 con 0x7f3058072a40 2023-11-04T05:28:01.850 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3054013470 con 0x7f3058072a40 2023-11-04T05:28:01.850 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.836+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f305401ccb0 con 0x7f3058072a40 2023-11-04T05:28:01.850 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.837+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3058074df0 con 0x7f3058072a40 2023-11-04T05:28:01.851 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.839+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f3054021060 con 0x7f3058072a40 2023-11-04T05:28:01.899 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:01.897+0000 7f866aa6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-04T05:28:01.912 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.911+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30580b6360 con 0x7f3058072a40 2023-11-04T05:28:01.912 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.911+0000 7f304effd700 1 -- 172.21.15.3:0/1749604858 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f305402a090 con 0x7f3058072a40 2023-11-04T05:28:01.913 INFO:teuthology.misc.health.smithi003.stdout: 2023-11-04T05:28:01.913 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:00.589881+0000","last_up_change":"0.000000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:01.915 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 msgr2=0x7f305815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:01.915 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 0x7f305815d6d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f305400ebe0 tx=0x7f305400eef0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.915 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 shutdown_connections 2023-11-04T05:28:01.916 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f305815a850 0x7f30580ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.916 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3058072a40 0x7f305815d6d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.916 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 --2- 172.21.15.3:0/1749604858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3058072ec0 0x7f3058152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:01.916 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 >> 172.21.15.3:0/1749604858 conn(0x7f30580cd170 msgr2=0x7f30580b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:01.917 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 shutdown_connections 2023-11-04T05:28:01.917 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:01.912+0000 7f30607f8700 1 -- 172.21.15.3:0/1749604858 wait complete. 2023-11-04T05:28:01.924 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-04T05:28:02.031 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:02.029+0000 7f866aa6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-04T05:28:02.076 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:02.075+0000 7fafbdd93200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-04T05:28:02.251 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:02.250+0000 7f866aa6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-04T05:28:02.295 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:02.294+0000 7fafbdd93200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-04T05:28:02.406 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:02.404+0000 7f866aa6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-04T05:28:02.435 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:02.434+0000 7fafbdd93200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-04T05:28:02.520 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:02.518+0000 7f866aa6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-04T05:28:02.664 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:02.663+0000 7fafbdd93200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-04T05:28:02.826 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:02.825+0000 7fafbdd93200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-04T05:28:02.844 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:02.842+0000 7f866aa6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-04T05:28:02.944 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:02.942+0000 7f866aa6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-04T05:28:02.947 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:02.947+0000 7fafbdd93200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-04T05:28:03.130 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:03.128+0000 7f866aa6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-04T05:28:03.280 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:03.279+0000 7fafbdd93200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-04T05:28:03.384 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:03.383+0000 7fafbdd93200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-04T05:28:03.578 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:03.577+0000 7fafbdd93200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-04T05:28:03.724 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:03.722+0000 7f866aa6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-04T05:28:04.322 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-11-04T05:28:04.321+0000 7fafbdd93200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-04T05:28:04.559 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T05:28:04.557+0000 7eff32107640 -1 osd.4 0 log_to_monitors true 2023-11-04T05:28:04.750 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:04.748+0000 7f86229cf700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-04T05:28:04.751 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-04T05:28:04.749+0000 7f86229cf700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-04T05:28:04.771 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T05:28:04.770+0000 7f1f78a01640 -1 osd.2 0 log_to_monitors true 2023-11-04T05:28:04.789 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T05:28:04.788+0000 7f58b5cc8640 -1 osd.0 0 log_to_monitors true 2023-11-04T05:28:04.798 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T05:28:04.798+0000 7f7dc5152640 -1 osd.1 0 log_to_monitors true 2023-11-04T05:28:04.815 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T05:28:04.814+0000 7f98005cd640 -1 osd.3 0 log_to_monitors true 2023-11-04T05:28:04.827 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T05:28:04.825+0000 7fdae79e3640 -1 osd.7 0 log_to_monitors true 2023-11-04T05:28:04.827 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T05:28:04.825+0000 7f8a33c55640 -1 osd.6 0 log_to_monitors true 2023-11-04T05:28:04.835 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T05:28:04.834+0000 7f104f806640 -1 osd.5 0 log_to_monitors true 2023-11-04T05:28:06.573 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T05:28:06.571+0000 7eff264bf700 -1 osd.4 0 waiting for initial osdmap 2023-11-04T05:28:06.590 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T05:28:06.589+0000 7eff1e09e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:07.132 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T05:28:07.131+0000 7f58aa080700 -1 osd.0 0 waiting for initial osdmap 2023-11-04T05:28:07.418 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T05:28:07.417+0000 7f58a1c5f700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:07.544 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T05:28:07.543+0000 7f1f6cdb9700 -1 osd.2 0 waiting for initial osdmap 2023-11-04T05:28:07.554 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T05:28:07.553+0000 7f1f64998700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:07.999 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T05:28:07.998+0000 7f97f4985700 -1 osd.3 0 waiting for initial osdmap 2023-11-04T05:28:08.005 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T05:28:08.005+0000 7f7db950a700 -1 osd.1 0 waiting for initial osdmap 2023-11-04T05:28:08.014 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T05:28:08.013+0000 7f97ec564700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:08.020 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T05:28:08.020+0000 7f7db10e9700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:08.350 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T05:28:08.348+0000 7f8a2800d700 -1 osd.6 0 waiting for initial osdmap 2023-11-04T05:28:08.365 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T05:28:08.364+0000 7f8a1fbec700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:08.375 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T05:28:08.374+0000 7f1043bbe700 -1 osd.5 0 waiting for initial osdmap 2023-11-04T05:28:08.391 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T05:28:08.390+0000 7f103b79d700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:08.663 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T05:28:08.662+0000 7fdadbd9b700 -1 osd.7 0 waiting for initial osdmap 2023-11-04T05:28:08.685 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T05:28:08.684+0000 7fdad397a700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-04T05:28:08.729 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:08.860 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.859+0000 7f123e3ef700 1 Processor -- start 2023-11-04T05:28:08.861 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.859+0000 7f123e3ef700 1 -- start start 2023-11-04T05:28:08.863 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f123e3ef700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f123815b880 0x7f123815f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:08.863 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f123e3ef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123815fa80 con 0x7f123815bc60 2023-11-04T05:28:08.863 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f123e3ef700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123815fd20 con 0x7f1238071ff0 2023-11-04T05:28:08.863 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f123e3ef700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123815c2d0 con 0x7f123815b880 2023-11-04T05:28:08.864 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f1237fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f123815b880 0x7f123815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:08.864 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f12377fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f123815bc60 0x7f123815c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56524/0 (socket says 172.21.15.3:56524) 2023-11-04T05:28:08.864 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f12377fe700 1 -- 172.21.15.3:0/846897364 learned_addr learned my addr 172.21.15.3:0/846897364 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:08.864 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f1237fff700 1 -- 172.21.15.3:0/846897364 >> v1:172.21.15.188:6789/0 conn(0x7f1238071ff0 legacy=0x7f123806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:08.864 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f1237fff700 1 -- 172.21.15.3:0/846897364 >> v1:172.21.15.3:6789/0 conn(0x7f123815bc60 legacy=0x7f123815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:08.865 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f1237fff700 1 -- 172.21.15.3:0/846897364 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238160000 con 0x7f123815b880 2023-11-04T05:28:08.865 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.860+0000 7f1236ffd700 1 -- 172.21.15.3:0/846897364 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011161465 0 0) 0x7f123815fa80 con 0x7f123815bc60 2023-11-04T05:28:08.865 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f1237fff700 1 --2- 172.21.15.3:0/846897364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815b880 0x7f123815f540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1220009840 tx=0x7f1220009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa67783a54728366 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:08.865 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f1236ffd700 1 -- 172.21.15.3:0/846897364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011830 con 0x7f123815b880 2023-11-04T05:28:08.865 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f1236ffd700 1 -- 172.21.15.3:0/846897364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220011990 con 0x7f123815b880 2023-11-04T05:28:08.866 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f1236ffd700 1 -- 172.21.15.3:0/846897364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011d20 con 0x7f123815b880 2023-11-04T05:28:08.866 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 -- 172.21.15.3:0/846897364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815b880 msgr2=0x7f123815f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:08.866 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 --2- 172.21.15.3:0/846897364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815b880 0x7f123815f540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1220009840 tx=0x7f1220009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:08.866 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 -- 172.21.15.3:0/846897364 shutdown_connections 2023-11-04T05:28:08.866 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 --2- 172.21.15.3:0/846897364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815b880 0x7f123815f540 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:08.867 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 -- 172.21.15.3:0/846897364 >> 172.21.15.3:0/846897364 conn(0x7f12380cd170 msgr2=0x7f12380cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:08.867 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 -- 172.21.15.3:0/846897364 shutdown_connections 2023-11-04T05:28:08.867 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.861+0000 7f123e3ef700 1 -- 172.21.15.3:0/846897364 wait complete. 2023-11-04T05:28:08.870 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.863+0000 7f123e3ef700 1 Processor -- start 2023-11-04T05:28:08.870 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.863+0000 7f123e3ef700 1 -- start start 2023-11-04T05:28:08.870 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1238071ff0 0x7f1238157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:08.870 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f123815b880 0x7f12380cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:08.871 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 0x7f12380c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:08.871 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380c7090 con 0x7f123815bc60 2023-11-04T05:28:08.871 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380ca3f0 con 0x7f1238071ff0 2023-11-04T05:28:08.871 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380ca690 con 0x7f123815b880 2023-11-04T05:28:08.872 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 0x7f12380c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:08.872 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 0x7f12380c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51184/0 (socket says 172.21.15.3:51184) 2023-11-04T05:28:08.872 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 -- 172.21.15.3:0/3894070165 learned_addr learned my addr 172.21.15.3:0/3894070165 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:08.872 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f1237fff700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1238071ff0 0x7f1238157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:08.872 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f12377fe700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f123815b880 0x7f12380cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:08.873 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 -- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f123815b880 msgr2=0x7f12380cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:08.873 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f123815b880 0x7f12380cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:08.873 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 -- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1238071ff0 msgr2=0x7f1238157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:08.873 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1238071ff0 0x7f1238157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:08.873 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 -- 172.21.15.3:0/3894070165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200094a0 con 0x7f123815bc60 2023-11-04T05:28:08.873 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f1237fff700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1238071ff0 0x7f1238157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:08.874 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123c98c700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 0x7f12380c8270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f123000cb50 tx=0x7f123000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:08.874 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230003c00 con 0x7f123815bc60 2023-11-04T05:28:08.874 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1230003d60 con 0x7f123815bc60 2023-11-04T05:28:08.874 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12380ca970 con 0x7f123815bc60 2023-11-04T05:28:08.875 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.864+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000ab70 con 0x7f123815bc60 2023-11-04T05:28:08.875 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.865+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12380caf30 con 0x7f123815bc60 2023-11-04T05:28:08.875 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.865+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12380cdf30 con 0x7f123815bc60 2023-11-04T05:28:08.875 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.866+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f123000acd0 con 0x7f123815bc60 2023-11-04T05:28:08.875 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.866+0000 7f1234ff9700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f121806ae70 0x7f121806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:08.876 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.866+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3631+0+0 (secure 0 0 0) 0x7f1230088d10 con 0x7f123815bc60 2023-11-04T05:28:08.876 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.866+0000 7f1237fff700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f121806ae70 0x7f121806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:08.876 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.867+0000 7f1237fff700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f121806ae70 0x7f121806d330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f122001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:08.876 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:08.871+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f12300546c0 con 0x7f123815bc60 2023-11-04T05:28:09.073 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.072+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f123806c9a0 con 0x7f123815bc60 2023-11-04T05:28:09.074 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.072+0000 7f1234ff9700 1 -- 172.21.15.3:0/3894070165 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9425 (secure 0 0 0) 0x7f1230057ce0 con 0x7f123815bc60 2023-11-04T05:28:09.074 INFO:teuthology.misc.health.smithi003.stdout: 2023-11-04T05:28:09.074 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":14,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:08.328759+0000","last_up_change":"2023-11-04T05:28:08.328759+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:09.076 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f121806ae70 msgr2=0x7f121806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:09.077 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f121806ae70 0x7f121806d330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f122001c040 comp rx=0 tx=0).stop 2023-11-04T05:28:09.077 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 msgr2=0x7f12380c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:09.077 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 0x7f12380c8270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f123000cb50 tx=0x7f123000cf10 comp rx=0 tx=0).stop 2023-11-04T05:28:09.077 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 shutdown_connections 2023-11-04T05:28:09.077 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f121806ae70 0x7f121806d330 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:09.078 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1238071ff0 0x7f1238157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:09.078 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f123815bc60 0x7f12380c8270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:09.078 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 --2- 172.21.15.3:0/3894070165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f123815b880 0x7f12380cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:09.078 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.074+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 >> 172.21.15.3:0/3894070165 conn(0x7f12380cd170 msgr2=0x7f1238072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:09.079 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.075+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 shutdown_connections 2023-11-04T05:28:09.079 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:09.075+0000 7f123e3ef700 1 -- 172.21.15.3:0/3894070165 wait complete. 2023-11-04T05:28:09.087 DEBUG:teuthology.misc:5 of 8 OSDs are up 2023-11-04T05:28:15.893 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:16.021 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.018+0000 7f4e75d8d700 1 Processor -- start 2023-11-04T05:28:16.021 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.019+0000 7f4e75d8d700 1 -- start start 2023-11-04T05:28:16.023 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.019+0000 7f4e75d8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e70072a40 0x7f4e700b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.023 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.019+0000 7f4e75d8d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e70072ef0 0x7f4e700b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.024 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.019+0000 7f4e75d8d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e700cbf30 con 0x7f4e7015a850 2023-11-04T05:28:16.024 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.019+0000 7f4e75d8d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700d03d0 con 0x7f4e70072a40 2023-11-04T05:28:16.024 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.019+0000 7f4e75d8d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700b3c30 con 0x7f4e70072ef0 2023-11-04T05:28:16.024 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e70072a40 0x7f4e700b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.025 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e70072a40 0x7f4e700b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51270/0 (socket says 172.21.15.3:51270) 2023-11-04T05:28:16.025 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 -- 172.21.15.3:0/3156103649 learned_addr learned my addr 172.21.15.3:0/3156103649 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:16.025 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6ffff700 1 --2- 172.21.15.3:0/3156103649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e70072ef0 0x7f4e700b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.025 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 -- 172.21.15.3:0/3156103649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e70072ef0 msgr2=0x7f4e700b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:16.025 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 --2- 172.21.15.3:0/3156103649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e70072ef0 0x7f4e700b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.026 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 -- 172.21.15.3:0/3156103649 >> v1:172.21.15.188:6789/0 conn(0x7f4e7015a850 legacy=0x7f4e700b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.026 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 -- 172.21.15.3:0/3156103649 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700cf850 con 0x7f4e70072a40 2023-11-04T05:28:16.026 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6effd700 1 --2- 172.21.15.3:0/3156103649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072a40 0x7f4e700b2170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400bbc0 tx=0x7f4e6400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4856d1c557f603 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.026 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6e7fc700 1 -- 172.21.15.3:0/3156103649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400dd50 con 0x7f4e70072a40 2023-11-04T05:28:16.026 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6e7fc700 1 -- 172.21.15.3:0/3156103649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64010470 con 0x7f4e70072a40 2023-11-04T05:28:16.026 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.020+0000 7f4e6e7fc700 1 -- 172.21.15.3:0/3156103649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64010800 con 0x7f4e70072a40 2023-11-04T05:28:16.027 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 -- 172.21.15.3:0/3156103649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072a40 msgr2=0x7f4e700b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.027 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/3156103649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072a40 0x7f4e700b2170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400bbc0 tx=0x7f4e6400bed0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.027 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 -- 172.21.15.3:0/3156103649 shutdown_connections 2023-11-04T05:28:16.027 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/3156103649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e70072ef0 0x7f4e700b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.027 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/3156103649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072a40 0x7f4e700b2170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.028 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 -- 172.21.15.3:0/3156103649 >> 172.21.15.3:0/3156103649 conn(0x7f4e700cd170 msgr2=0x7f4e700cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:16.028 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 -- 172.21.15.3:0/3156103649 shutdown_connections 2023-11-04T05:28:16.028 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.021+0000 7f4e75d8d700 1 -- 172.21.15.3:0/3156103649 wait complete. 2023-11-04T05:28:16.031 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.022+0000 7f4e75d8d700 1 Processor -- start 2023-11-04T05:28:16.031 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.023+0000 7f4e75d8d700 1 -- start start 2023-11-04T05:28:16.031 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e75d8d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e70072a40 0x7f4e700c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.032 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e75d8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 0x7f4e7015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.032 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e75d8d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e7015a850 0x7f4e70158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.032 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e75d8d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015d170 con 0x7f4e70072ef0 2023-11-04T05:28:16.032 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e75d8d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015c5a0 con 0x7f4e70072a40 2023-11-04T05:28:16.033 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e75d8d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015c840 con 0x7f4e7015a850 2023-11-04T05:28:16.033 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 0x7f4e7015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.033 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 0x7f4e7015d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51276/0 (socket says 172.21.15.3:51276) 2023-11-04T05:28:16.033 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 -- 172.21.15.3:0/107230583 learned_addr learned my addr 172.21.15.3:0/107230583 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:16.033 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6ffff700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e7015a850 0x7f4e70158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.034 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6f7fe700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e70072a40 0x7f4e700c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.034 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 -- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e7015a850 msgr2=0x7f4e70158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.034 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e7015a850 0x7f4e70158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.034 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 -- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e70072a40 msgr2=0x7f4e700c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.034 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e70072a40 0x7f4e700c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.035 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 -- 172.21.15.3:0/107230583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6400b7f0 con 0x7f4e70072ef0 2023-11-04T05:28:16.035 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.024+0000 7f4e6effd700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 0x7f4e7015d600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.035 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.025+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6401f070 con 0x7f4e70072ef0 2023-11-04T05:28:16.035 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.025+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e640038b0 con 0x7f4e70072ef0 2023-11-04T05:28:16.035 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.025+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7015cb20 con 0x7f4e70072ef0 2023-11-04T05:28:16.036 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.025+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64003c40 con 0x7f4e70072ef0 2023-11-04T05:28:16.036 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.025+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e70159370 con 0x7f4e70072ef0 2023-11-04T05:28:16.036 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.025+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7015d2e0 con 0x7f4e70072ef0 2023-11-04T05:28:16.036 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.026+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e6402c430 con 0x7f4e70072ef0 2023-11-04T05:28:16.036 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.027+0000 7f4e57fff700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4e5c06af00 0x7f4e5c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.037 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.027+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e6408c540 con 0x7f4e70072ef0 2023-11-04T05:28:16.037 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.027+0000 7f4e6f7fe700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4e5c06af00 0x7f4e5c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.037 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.027+0000 7f4e6f7fe700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4e5c06af00 0x7f4e5c06d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e60007fd0 tx=0x7f4e60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.037 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.032+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4e640578b0 con 0x7f4e70072ef0 2023-11-04T05:28:16.235 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.233+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e70074df0 con 0x7f4e70072ef0 2023-11-04T05:28:16.236 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.234+0000 7f4e57fff700 1 -- 172.21.15.3:0/107230583 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f4e6405aed0 con 0x7f4e70072ef0 2023-11-04T05:28:16.236 INFO:teuthology.misc.health.smithi003.stdout: 2023-11-04T05:28:16.237 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":18,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:12.385400+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:16.239 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4e5c06af00 msgr2=0x7f4e5c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.239 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4e5c06af00 0x7f4e5c06d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e60007fd0 tx=0x7f4e60009040 comp rx=0 tx=0).stop 2023-11-04T05:28:16.239 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 msgr2=0x7f4e7015d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.239 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 0x7f4e7015d600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e64004750 comp rx=0 tx=0).stop 2023-11-04T05:28:16.240 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 shutdown_connections 2023-11-04T05:28:16.240 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4e5c06af00 0x7f4e5c06d3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.240 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e70072a40 0x7f4e700c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.240 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e7015a850 0x7f4e70158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.240 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 --2- 172.21.15.3:0/107230583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e70072ef0 0x7f4e7015d600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.241 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.236+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 >> 172.21.15.3:0/107230583 conn(0x7f4e700cd170 msgr2=0x7f4e700b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:16.241 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.237+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 shutdown_connections 2023-11-04T05:28:16.241 INFO:teuthology.misc.health.smithi003.stderr:2023-11-04T05:28:16.237+0000 7f4e75d8d700 1 -- 172.21.15.3:0/107230583 wait complete. 2023-11-04T05:28:16.249 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-04T05:28:16.250 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-04T05:28:16.250 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-04T05:28:16.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-04T05:28:16.253 INFO:tasks.ceph.mds.a:Started 2023-11-04T05:28:16.253 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-04T05:28:16.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-04T05:28:16.256 INFO:tasks.ceph.mds.c:Started 2023-11-04T05:28:16.256 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-04T05:28:16.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-04T05:28:16.259 INFO:tasks.ceph.mds.b:Started 2023-11-04T05:28:16.260 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-04T05:28:16.260 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-04T05:28:16.262 INFO:tasks.ceph.mds.d:Started 2023-11-04T05:28:16.263 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-04T05:28:16.263 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:16.335 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-11-04T05:28:16.336 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-11-04T05:28:16.350 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-04T05:28:16.351 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-04T05:28:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.392+0000 7f64f107e700 1 Processor -- start 2023-11-04T05:28:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.392+0000 7f64f107e700 1 -- start start 2023-11-04T05:28:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64f107e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec1526c0 0x7f64ec0b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64f107e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f64ec15bd70 0x7f64ec0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64f107e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec15c250 con 0x7f64ec152aa0 2023-11-04T05:28:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64f107e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec15cdd0 con 0x7f64ec15bd70 2023-11-04T05:28:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64f107e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec15c530 con 0x7f64ec1526c0 2023-11-04T05:28:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ea59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f64ec152aa0 0x7f64ec0cde20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56656/0 (socket says 172.21.15.3:56656) 2023-11-04T05:28:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ea59c700 1 -- 172.21.15.3:0/3791309079 learned_addr learned my addr 172.21.15.3:0/3791309079 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64eb59e700 1 --2- 172.21.15.3:0/3791309079 >> v2:172.21.15.188:3300/0 conn(0x7f64ec15bd70 0x7f64ec0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ead9d700 1 --2- 172.21.15.3:0/3791309079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec1526c0 0x7f64ec0b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64e9d9b700 1 -- 172.21.15.3:0/3791309079 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002974680 0 0) 0x7f64ec15c250 con 0x7f64ec152aa0 2023-11-04T05:28:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64e9d9b700 1 -- 172.21.15.3:0/3791309079 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d40035b0 con 0x7f64ec152aa0 2023-11-04T05:28:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ead9d700 1 -- 172.21.15.3:0/3791309079 >> v2:172.21.15.188:3300/0 conn(0x7f64ec15bd70 msgr2=0x7f64ec0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ead9d700 1 --2- 172.21.15.3:0/3791309079 >> v2:172.21.15.188:3300/0 conn(0x7f64ec15bd70 0x7f64ec0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ead9d700 1 -- 172.21.15.3:0/3791309079 >> v1:172.21.15.3:6789/0 conn(0x7f64ec152aa0 legacy=0x7f64ec0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.393+0000 7f64ead9d700 1 -- 172.21.15.3:0/3791309079 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec15c810 con 0x7f64ec1526c0 2023-11-04T05:28:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64ead9d700 1 --2- 172.21.15.3:0/3791309079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec1526c0 0x7f64ec0b6ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009840 tx=0x7f64e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c2e9f9a6145462 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64e9d9b700 1 -- 172.21.15.3:0/3791309079 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0011830 con 0x7f64ec1526c0 2023-11-04T05:28:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64e9d9b700 1 -- 172.21.15.3:0/3791309079 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0011990 con 0x7f64ec1526c0 2023-11-04T05:28:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64e9d9b700 1 -- 172.21.15.3:0/3791309079 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0011d20 con 0x7f64ec1526c0 2023-11-04T05:28:16.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 -- 172.21.15.3:0/3791309079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec1526c0 msgr2=0x7f64ec0b6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 --2- 172.21.15.3:0/3791309079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec1526c0 0x7f64ec0b6ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009840 tx=0x7f64e0009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:16.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 -- 172.21.15.3:0/3791309079 shutdown_connections 2023-11-04T05:28:16.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 --2- 172.21.15.3:0/3791309079 >> v2:172.21.15.188:3300/0 conn(0x7f64ec15bd70 0x7f64ec0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 --2- 172.21.15.3:0/3791309079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec1526c0 0x7f64ec0b6ee0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 -- 172.21.15.3:0/3791309079 >> 172.21.15.3:0/3791309079 conn(0x7f64ec06ab20 msgr2=0x7f64ec06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:16.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 -- 172.21.15.3:0/3791309079 shutdown_connections 2023-11-04T05:28:16.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.394+0000 7f64f107e700 1 -- 172.21.15.3:0/3791309079 wait complete. 2023-11-04T05:28:16.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.396+0000 7f64f107e700 1 Processor -- start 2023-11-04T05:28:16.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.396+0000 7f64f107e700 1 -- start start 2023-11-04T05:28:16.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64f107e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ec1526c0 0x7f64ec153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64f107e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec15bd70 0x7f64ec1f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64f107e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 0x7f64ec14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64f107e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec14fa80 con 0x7f64ec14ec10 2023-11-04T05:28:16.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64f107e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1f5b80 con 0x7f64ec1526c0 2023-11-04T05:28:16.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64f107e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0cfc70 con 0x7f64ec15bd70 2023-11-04T05:28:16.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 0x7f64ec14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 0x7f64ec14e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51318/0 (socket says 172.21.15.3:51318) 2023-11-04T05:28:16.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 -- 172.21.15.3:0/2365680234 learned_addr learned my addr 172.21.15.3:0/2365680234 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:16.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64ea59c700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec15bd70 0x7f64ec1f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64ead9d700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ec1526c0 0x7f64ec153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 -- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec15bd70 msgr2=0x7f64ec1f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec15bd70 0x7f64ec1f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 -- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ec1526c0 msgr2=0x7f64ec153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ec1526c0 0x7f64ec153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 -- 172.21.15.3:0/2365680234 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00094a0 con 0x7f64ec14ec10 2023-11-04T05:28:16.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64eb59e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 0x7f64ec14e450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64e400efc0 tx=0x7f64e4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4009b20 con 0x7f64ec14ec10 2023-11-04T05:28:16.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.397+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400f040 con 0x7f64ec14ec10 2023-11-04T05:28:16.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.398+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0d0230 con 0x7f64ec14ec10 2023-11-04T05:28:16.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.398+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400cc90 con 0x7f64ec14ec10 2023-11-04T05:28:16.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.398+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0d1e90 con 0x7f64ec14ec10 2023-11-04T05:28:16.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.398+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec06b4b0 con 0x7f64ec14ec10 2023-11-04T05:28:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.399+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64e4014480 con 0x7f64ec14ec10 2023-11-04T05:28:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.399+0000 7f64d37fe700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f64d806af00 0x7f64d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.400+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f64e4089ca0 con 0x7f64ec14ec10 2023-11-04T05:28:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.400+0000 7f64ead9d700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f64d806af00 0x7f64d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.400+0000 7f64ead9d700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f64d806af00 0x7f64d806d3c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f64e0000c00 tx=0x7f64e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.404+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f64e40550a0 con 0x7f64ec14ec10 2023-11-04T05:28:16.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.607+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64ec0d03c0 con 0x7f64ec14ec10 2023-11-04T05:28:16.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.608+0000 7f64d37fe700 1 -- 172.21.15.3:0/2365680234 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f64e4018090 con 0x7f64ec14ec10 2023-11-04T05:28:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:16.610 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:12.385400+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:16.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f64d806af00 msgr2=0x7f64d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f64d806af00 0x7f64d806d3c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f64e0000c00 tx=0x7f64e0008030 comp rx=0 tx=0).stop 2023-11-04T05:28:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 msgr2=0x7f64ec14e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 0x7f64ec14e450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64e400efc0 tx=0x7f64e4009f60 comp rx=0 tx=0).stop 2023-11-04T05:28:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 shutdown_connections 2023-11-04T05:28:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f64d806af00 0x7f64d806d3c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ec1526c0 0x7f64ec153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec14ec10 0x7f64ec14e450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 --2- 172.21.15.3:0/2365680234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ec15bd70 0x7f64ec1f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 >> 172.21.15.3:0/2365680234 conn(0x7f64ec06ab20 msgr2=0x7f64ec0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 shutdown_connections 2023-11-04T05:28:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.610+0000 7f64f107e700 1 -- 172.21.15.3:0/2365680234 wait complete. 2023-11-04T05:28:16.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-04T05:28:16.625 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:28:16.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.750+0000 7f7620f1b700 1 Processor -- start 2023-11-04T05:28:16.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.750+0000 7f7620f1b700 1 -- start start 2023-11-04T05:28:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f7620f1b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f761c0cee80 0x7f761c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f7620f1b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f761c15b260 0x7f761c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f7620f1b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761c0ccc70 con 0x7f761c0ceaa0 2023-11-04T05:28:16.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f7620f1b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761c15baa0 con 0x7f761c0cee80 2023-11-04T05:28:16.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f7620f1b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761c0ccec0 con 0x7f761c15b260 2023-11-04T05:28:16.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f761a59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f761c0ceaa0 0x7f761c0cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56670/0 (socket says 172.21.15.3:56670) 2023-11-04T05:28:16.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f7619d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f761c0cee80 0x7f761c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.751+0000 7f761a59c700 1 -- 172.21.15.3:0/1779665589 learned_addr learned my addr 172.21.15.3:0/1779665589 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:16.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f761ad9d700 1 --2- 172.21.15.3:0/1779665589 >> v2:172.21.15.188:3300/0 conn(0x7f761c15b260 0x7f761c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7619d9b700 1 -- 172.21.15.3:0/1779665589 >> v2:172.21.15.188:3300/0 conn(0x7f761c15b260 msgr2=0x7f761c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:16.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7619d9b700 1 --2- 172.21.15.3:0/1779665589 >> v2:172.21.15.188:3300/0 conn(0x7f761c15b260 0x7f761c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7619d9b700 1 -- 172.21.15.3:0/1779665589 >> v1:172.21.15.3:6789/0 conn(0x7f761c0ceaa0 legacy=0x7f761c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7619d9b700 1 -- 172.21.15.3:0/1779665589 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c0cd1a0 con 0x7f761c0cee80 2023-11-04T05:28:16.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f761959a700 1 -- 172.21.15.3:0/1779665589 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464593915 0 0) 0x7f761c0ccc70 con 0x7f761c0ceaa0 2023-11-04T05:28:16.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f761ad9d700 1 --2- 172.21.15.3:0/1779665589 >> v2:172.21.15.188:3300/0 conn(0x7f761c15b260 0x7f761c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:16.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7619d9b700 1 --2- 172.21.15.3:0/1779665589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c15ad20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f761c0d00e0 tx=0x7f7610009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dbbc9aef0a9b7c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f761959a700 1 -- 172.21.15.3:0/1779665589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011830 con 0x7f761c0cee80 2023-11-04T05:28:16.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f761959a700 1 -- 172.21.15.3:0/1779665589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7610011990 con 0x7f761c0cee80 2023-11-04T05:28:16.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f761959a700 1 -- 172.21.15.3:0/1779665589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011d20 con 0x7f761c0cee80 2023-11-04T05:28:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7620f1b700 1 -- 172.21.15.3:0/1779665589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 msgr2=0x7f761c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.752+0000 7f7620f1b700 1 --2- 172.21.15.3:0/1779665589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c15ad20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f761c0d00e0 tx=0x7f7610009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.753+0000 7f7620f1b700 1 -- 172.21.15.3:0/1779665589 shutdown_connections 2023-11-04T05:28:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.753+0000 7f7620f1b700 1 --2- 172.21.15.3:0/1779665589 >> v2:172.21.15.188:3300/0 conn(0x7f761c15b260 0x7f761c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.753+0000 7f7620f1b700 1 --2- 172.21.15.3:0/1779665589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c15ad20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.753+0000 7f7620f1b700 1 -- 172.21.15.3:0/1779665589 >> 172.21.15.3:0/1779665589 conn(0x7f761c072560 msgr2=0x7f761c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.753+0000 7f7620f1b700 1 -- 172.21.15.3:0/1779665589 shutdown_connections 2023-11-04T05:28:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.753+0000 7f7620f1b700 1 -- 172.21.15.3:0/1779665589 wait complete. 2023-11-04T05:28:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.754+0000 7f7620f1b700 1 Processor -- start 2023-11-04T05:28:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.754+0000 7f7620f1b700 1 -- start start 2023-11-04T05:28:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.755+0000 7f7620f1b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761c0ceaa0 0x7f761c1f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.755+0000 7f7620f1b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.755+0000 7f7620f1b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761c15b260 0x7f761c0c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.755+0000 7f7620f1b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c0c72b0 con 0x7f761c0cee80 2023-11-04T05:28:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.755+0000 7f7620f1b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c0b2d20 con 0x7f761c0ceaa0 2023-11-04T05:28:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.755+0000 7f7620f1b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c0c4180 con 0x7f761c15b260 2023-11-04T05:28:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c1564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51330/0 (socket says 172.21.15.3:51330) 2023-11-04T05:28:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 -- 172.21.15.3:0/3240429603 learned_addr learned my addr 172.21.15.3:0/3240429603 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f761a59c700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761c0ceaa0 0x7f761c1f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f761ad9d700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761c15b260 0x7f761c0c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 -- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761c15b260 msgr2=0x7f761c0c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761c15b260 0x7f761c0c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 -- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761c0ceaa0 msgr2=0x7f761c1f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761c0ceaa0 0x7f761c1f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 -- 172.21.15.3:0/3240429603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100094a0 con 0x7f761c0cee80 2023-11-04T05:28:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7619d9b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c1564d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f76100090b0 tx=0x7f7610004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100036a0 con 0x7f761c0cee80 2023-11-04T05:28:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761001f070 con 0x7f761c0cee80 2023-11-04T05:28:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c1f4900 con 0x7f761c0cee80 2023-11-04T05:28:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610003a40 con 0x7f761c0cee80 2023-11-04T05:28:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.756+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c0c43d0 con 0x7f761c0cee80 2023-11-04T05:28:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.757+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c074df0 con 0x7f761c0cee80 2023-11-04T05:28:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.758+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7610003ba0 con 0x7f761c0cee80 2023-11-04T05:28:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.758+0000 7f760affd700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f75fc06af00 0x7f75fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.758+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7610021070 con 0x7f761c0cee80 2023-11-04T05:28:16.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.758+0000 7f761a59c700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f75fc06af00 0x7f75fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:16.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.759+0000 7f761a59c700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f75fc06af00 0x7f75fc06d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f760c007fd0 tx=0x7f760c007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:16.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.763+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7610055f30 con 0x7f761c0cee80 2023-11-04T05:28:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.963+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f761c0ce6e0 con 0x7f761c0cee80 2023-11-04T05:28:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.963+0000 7f760affd700 1 -- 172.21.15.3:0/3240429603 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76100150a0 con 0x7f761c0cee80 2023-11-04T05:28:16.966 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:28:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f75fc06af00 msgr2=0x7f75fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f75fc06af00 0x7f75fc06d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f760c007fd0 tx=0x7f760c007de0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 msgr2=0x7f761c1564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c1564d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f76100090b0 tx=0x7f7610004750 comp rx=0 tx=0).stop 2023-11-04T05:28:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 shutdown_connections 2023-11-04T05:28:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f75fc06af00 0x7f75fc06d3c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761c0ceaa0 0x7f761c1f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761c15b260 0x7f761c0c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 --2- 172.21.15.3:0/3240429603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f761c0cee80 0x7f761c1564d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 >> 172.21.15.3:0/3240429603 conn(0x7f761c072560 msgr2=0x7f761c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 shutdown_connections 2023-11-04T05:28:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:16.965+0000 7f7620f1b700 1 -- 172.21.15.3:0/3240429603 wait complete. 2023-11-04T05:28:16.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.103+0000 7f8172852700 1 Processor -- start 2023-11-04T05:28:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.103+0000 7f8172852700 1 -- start start 2023-11-04T05:28:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f8172852700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c0cf330 0x7f816c0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f8172852700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c15af70 con 0x7f816c15b400 2023-11-04T05:28:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f8172852700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c072d30 con 0x7f816c0cf710 2023-11-04T05:28:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f8172852700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c0cde10 con 0x7f816c0cf330 2023-11-04T05:28:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f8170def700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f816c15b400 0x7f816c0cbb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56688/0 (socket says 172.21.15.3:56688) 2023-11-04T05:28:17.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f8170def700 1 -- 172.21.15.3:0/1866944550 learned_addr learned my addr 172.21.15.3:0/1866944550 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:17.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816bfff700 1 --2- 172.21.15.3:0/1866944550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c0cf330 0x7f816c0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291181568 0 0) 0x7f816c15af70 con 0x7f816c15b400 2023-11-04T05:28:17.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81540035b0 con 0x7f816c15b400 2023-11-04T05:28:17.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394442836 0 0) 0x7f816c072d30 con 0x7f816c0cf710 2023-11-04T05:28:17.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c15af70 con 0x7f816c0cf710 2023-11-04T05:28:17.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816bfff700 1 -- 172.21.15.3:0/1866944550 >> v1:172.21.15.188:6789/0 conn(0x7f816c0cf710 legacy=0x7f816c0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816bfff700 1 -- 172.21.15.3:0/1866944550 >> v1:172.21.15.3:6789/0 conn(0x7f816c15b400 legacy=0x7f816c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190852018 0 0) 0x7f81540035b0 con 0x7f816c15b400 2023-11-04T05:28:17.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.104+0000 7f816bfff700 1 -- 172.21.15.3:0/1866944550 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0cd570 con 0x7f816c0cf330 2023-11-04T05:28:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f816bfff700 1 --2- 172.21.15.3:0/1866944550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c0cf330 0x7f816c0ccf50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8160009870 tx=0x7f8160009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a11b9101ad559da server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640019f0 con 0x7f816c0cf330 2023-11-04T05:28:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160004030 con 0x7f816c0cf330 2023-11-04T05:28:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f816affd700 1 -- 172.21.15.3:0/1866944550 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81600043a0 con 0x7f816c0cf330 2023-11-04T05:28:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 -- 172.21.15.3:0/1866944550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c0cf330 msgr2=0x7f816c0ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 --2- 172.21.15.3:0/1866944550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c0cf330 0x7f816c0ccf50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8160009870 tx=0x7f8160009b80 comp rx=0 tx=0).stop 2023-11-04T05:28:17.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 -- 172.21.15.3:0/1866944550 shutdown_connections 2023-11-04T05:28:17.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 --2- 172.21.15.3:0/1866944550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c0cf330 0x7f816c0ccf50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 -- 172.21.15.3:0/1866944550 >> 172.21.15.3:0/1866944550 conn(0x7f816c06df90 msgr2=0x7f816c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:17.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 -- 172.21.15.3:0/1866944550 shutdown_connections 2023-11-04T05:28:17.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.105+0000 7f8172852700 1 -- 172.21.15.3:0/1866944550 wait complete. 2023-11-04T05:28:17.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 Processor -- start 2023-11-04T05:28:17.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 -- start start 2023-11-04T05:28:17.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 0x7f816c14b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f816c0cf710 0x7f816c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c15b400 0x7f816c1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c148c30 con 0x7f816c0cf330 2023-11-04T05:28:17.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c149a70 con 0x7f816c0cf710 2023-11-04T05:28:17.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.107+0000 7f8172852700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c148e60 con 0x7f816c15b400 2023-11-04T05:28:17.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 0x7f816c14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 0x7f816c14b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51342/0 (socket says 172.21.15.3:51342) 2023-11-04T05:28:17.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 -- 172.21.15.3:0/859298691 learned_addr learned my addr 172.21.15.3:0/859298691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:17.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816b7fe700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f816c0cf710 0x7f816c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f8170def700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c15b400 0x7f816c1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 -- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c15b400 msgr2=0x7f816c1486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c15b400 0x7f816c1486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 -- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f816c0cf710 msgr2=0x7f816c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f816c0cf710 0x7f816c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 -- 172.21.15.3:0/859298691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81600094a0 con 0x7f816c0cf330 2023-11-04T05:28:17.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f816bfff700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 0x7f816c14b330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81600097c0 tx=0x7f8160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160003ad0 con 0x7f816c0cf330 2023-11-04T05:28:17.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816001f070 con 0x7f816c0cf330 2023-11-04T05:28:17.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c149140 con 0x7f816c0cf330 2023-11-04T05:28:17.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160003e50 con 0x7f816c0cf330 2023-11-04T05:28:17.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.108+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c149700 con 0x7f816c0cf330 2023-11-04T05:28:17.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.109+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816c0b5980 con 0x7f816c0cf330 2023-11-04T05:28:17.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.110+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8160021030 con 0x7f816c0cf330 2023-11-04T05:28:17.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.110+0000 7f8168ff9700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f815806af00 0x7f815806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.110+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f816008a400 con 0x7f816c0cf330 2023-11-04T05:28:17.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.111+0000 7f816b7fe700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f815806af00 0x7f815806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.111+0000 7f816b7fe700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f815806af00 0x7f815806d3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f816c155410 tx=0x7f815c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.115+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f81600559b0 con 0x7f816c0cf330 2023-11-04T05:28:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.362+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f816c1498c0 con 0x7f816c0cf330 2023-11-04T05:28:17.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.362+0000 7f8168ff9700 1 -- 172.21.15.3:0/859298691 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f8160058fd0 con 0x7f816c0cf330 2023-11-04T05:28:17.365 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:17.365 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-04T05:28:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f815806af00 msgr2=0x7f815806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f815806af00 0x7f815806d3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f816c155410 tx=0x7f815c009040 comp rx=0 tx=0).stop 2023-11-04T05:28:17.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 msgr2=0x7f816c14b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 0x7f816c14b330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81600097c0 tx=0x7f8160004750 comp rx=0 tx=0).stop 2023-11-04T05:28:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 shutdown_connections 2023-11-04T05:28:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f815806af00 0x7f815806d3c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f816c15b400 0x7f816c1486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f816c0cf330 0x7f816c14b330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 --2- 172.21.15.3:0/859298691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f816c0cf710 0x7f816c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 >> 172.21.15.3:0/859298691 conn(0x7f816c06df90 msgr2=0x7f816c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 shutdown_connections 2023-11-04T05:28:17.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.365+0000 7f8172852700 1 -- 172.21.15.3:0/859298691 wait complete. 2023-11-04T05:28:17.370 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2023-11-04T05:28:17.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.502+0000 7f5e3d994700 1 Processor -- start 2023-11-04T05:28:17.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.502+0000 7f5e3d994700 1 -- start start 2023-11-04T05:28:17.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e3d994700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e380d2690 0x7f5e3806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e3d994700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e380b2e00 0x7f5e380d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e3d994700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e381616a0 con 0x7f5e381612c0 2023-11-04T05:28:17.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e3d994700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380b2bb0 con 0x7f5e380b2e00 2023-11-04T05:28:17.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e3d994700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380d42c0 con 0x7f5e380d2690 2023-11-04T05:28:17.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e380b2e00 0x7f5e380d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e36ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5e381612c0 0x7f5e380d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56700/0 (socket says 172.21.15.3:56700) 2023-11-04T05:28:17.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e36ffd700 1 -- 172.21.15.3:0/2199573902 learned_addr learned my addr 172.21.15.3:0/2199573902 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:17.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e367fc700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e380d2690 0x7f5e3806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e377fe700 1 -- 172.21.15.3:0/2199573902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e380d2690 msgr2=0x7f5e3806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:17.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e377fe700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e380d2690 0x7f5e3806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e377fe700 1 -- 172.21.15.3:0/2199573902 >> v1:172.21.15.3:6789/0 conn(0x7f5e381612c0 legacy=0x7f5e380d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e377fe700 1 -- 172.21.15.3:0/2199573902 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380d45a0 con 0x7f5e380b2e00 2023-11-04T05:28:17.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e35ffb700 1 -- 172.21.15.3:0/2199573902 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549858410 0 0) 0x7f5e381616a0 con 0x7f5e381612c0 2023-11-04T05:28:17.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.503+0000 7f5e367fc700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e380d2690 0x7f5e3806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:17.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e377fe700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380b2e00 0x7f5e380d1780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009d90 tx=0x7f5e3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3043daa6ac811c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e35ffb700 1 -- 172.21.15.3:0/2199573902 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000e070 con 0x7f5e380b2e00 2023-11-04T05:28:17.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e35ffb700 1 -- 172.21.15.3:0/2199573902 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3000bbd0 con 0x7f5e380b2e00 2023-11-04T05:28:17.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e35ffb700 1 -- 172.21.15.3:0/2199573902 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30012430 con 0x7f5e380b2e00 2023-11-04T05:28:17.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 -- 172.21.15.3:0/2199573902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380b2e00 msgr2=0x7f5e380d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380b2e00 0x7f5e380d1780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009d90 tx=0x7f5e3000b330 comp rx=0 tx=0).stop 2023-11-04T05:28:17.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 -- 172.21.15.3:0/2199573902 shutdown_connections 2023-11-04T05:28:17.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380b2e00 0x7f5e380d1780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2199573902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e380d2690 0x7f5e3806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 -- 172.21.15.3:0/2199573902 >> 172.21.15.3:0/2199573902 conn(0x7f5e3808aff0 msgr2=0x7f5e380d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:17.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 -- 172.21.15.3:0/2199573902 shutdown_connections 2023-11-04T05:28:17.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.504+0000 7f5e3d994700 1 -- 172.21.15.3:0/2199573902 wait complete. 2023-11-04T05:28:17.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.506+0000 7f5e3d994700 1 Processor -- start 2023-11-04T05:28:17.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.506+0000 7f5e3d994700 1 -- start start 2023-11-04T05:28:17.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.506+0000 7f5e3d994700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380b2e00 0x7f5e380ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.506+0000 7f5e3d994700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 0x7f5e380cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.506+0000 7f5e3d994700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e381612c0 0x7f5e381f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e3d994700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380b6db0 con 0x7f5e380d2690 2023-11-04T05:28:17.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e3d994700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380b8cf0 con 0x7f5e380b2e00 2023-11-04T05:28:17.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e3d994700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380b8490 con 0x7f5e381612c0 2023-11-04T05:28:17.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 0x7f5e380cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 0x7f5e380cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51360/0 (socket says 172.21.15.3:51360) 2023-11-04T05:28:17.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 -- 172.21.15.3:0/2961088988 learned_addr learned my addr 172.21.15.3:0/2961088988 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:17.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e377fe700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e381612c0 0x7f5e381f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e36ffd700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380b2e00 0x7f5e380ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 -- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e381612c0 msgr2=0x7f5e381f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e381612c0 0x7f5e381f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 -- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380b2e00 msgr2=0x7f5e380ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380b2e00 0x7f5e380ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 -- 172.21.15.3:0/2961088988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e30009a70 con 0x7f5e380d2690 2023-11-04T05:28:17.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e36ffd700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380b2e00 0x7f5e380ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:17.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e367fc700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 0x7f5e380cd4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c7f0 tx=0x7f5e2800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28011800 con 0x7f5e380d2690 2023-11-04T05:28:17.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28011960 con 0x7f5e380d2690 2023-11-04T05:28:17.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e380b8730 con 0x7f5e380d2690 2023-11-04T05:28:17.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28011d10 con 0x7f5e380d2690 2023-11-04T05:28:17.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.507+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e380cdb10 con 0x7f5e380d2690 2023-11-04T05:28:17.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.508+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e380cab00 con 0x7f5e380d2690 2023-11-04T05:28:17.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.509+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e28018360 con 0x7f5e380d2690 2023-11-04T05:28:17.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.509+0000 7f5e1ffff700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e2406af00 0x7f5e2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.509+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e280891b0 con 0x7f5e380d2690 2023-11-04T05:28:17.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.509+0000 7f5e36ffd700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e2406af00 0x7f5e2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.510+0000 7f5e36ffd700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e2406af00 0x7f5e2406d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c001730 tx=0x7f5e2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.514+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5e380cab00 con 0x7f5e380d2690 2023-11-04T05:28:17.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.714+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e380cab00 con 0x7f5e380d2690 2023-11-04T05:28:17.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.715+0000 7f5e1ffff700 1 -- 172.21.15.3:0/2961088988 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f5e380cab00 con 0x7f5e380d2690 2023-11-04T05:28:17.717 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:17.718 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:12.385400+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:17.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e2406af00 msgr2=0x7f5e2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e2406af00 0x7f5e2406d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c001730 tx=0x7f5e2c009040 comp rx=0 tx=0).stop 2023-11-04T05:28:17.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 msgr2=0x7f5e380cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 0x7f5e380cd4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c7f0 tx=0x7f5e2800cbb0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 shutdown_connections 2023-11-04T05:28:17.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e2406af00 0x7f5e2406d3c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380b2e00 0x7f5e380ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e381612c0 0x7f5e381f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 --2- 172.21.15.3:0/2961088988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e380d2690 0x7f5e380cd4d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 >> 172.21.15.3:0/2961088988 conn(0x7f5e3808aff0 msgr2=0x7f5e380d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 shutdown_connections 2023-11-04T05:28:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.717+0000 7f5e3d994700 1 -- 172.21.15.3:0/2961088988 wait complete. 2023-11-04T05:28:17.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-04T05:28:17.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:28:17.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.858+0000 7f45ddd84700 1 Processor -- start 2023-11-04T05:28:17.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.858+0000 7f45ddd84700 1 -- start start 2023-11-04T05:28:17.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45ddd84700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d80cc100 0x7f45d815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45ddd84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45d81601a0 0x7f45d806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45ddd84700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d80b0f20 con 0x7f45d8155df0 2023-11-04T05:28:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45ddd84700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d815c560 con 0x7f45d81601a0 2023-11-04T05:28:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45ddd84700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80cc560 con 0x7f45d80cc100 2023-11-04T05:28:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45d7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45d81601a0 0x7f45d806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45d7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f45d81601a0 0x7f45d806ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51370/0 (socket says 172.21.15.3:51370) 2023-11-04T05:28:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45d7fff700 1 -- 172.21.15.3:0/3028850912 learned_addr learned my addr 172.21.15.3:0/3028850912 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.859+0000 7f45d6ffd700 1 --2- 172.21.15.3:0/3028850912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d80cc100 0x7f45d815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d7fff700 1 -- 172.21.15.3:0/3028850912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d80cc100 msgr2=0x7f45d815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d7fff700 1 --2- 172.21.15.3:0/3028850912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d80cc100 0x7f45d815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d7fff700 1 -- 172.21.15.3:0/3028850912 >> v1:172.21.15.188:6789/0 conn(0x7f45d8155df0 legacy=0x7f45d815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d7fff700 1 -- 172.21.15.3:0/3028850912 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d815f4a0 con 0x7f45d81601a0 2023-11-04T05:28:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d7fff700 1 --2- 172.21.15.3:0/3028850912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d81601a0 0x7f45d806ab20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009e50 tx=0x7f45d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2a0ff4b6546f51 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d67fc700 1 -- 172.21.15.3:0/3028850912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d000e070 con 0x7f45d81601a0 2023-11-04T05:28:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d67fc700 1 -- 172.21.15.3:0/3028850912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d000b9c0 con 0x7f45d81601a0 2023-11-04T05:28:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45d67fc700 1 -- 172.21.15.3:0/3028850912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d000bd70 con 0x7f45d81601a0 2023-11-04T05:28:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45ddd84700 1 -- 172.21.15.3:0/3028850912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d81601a0 msgr2=0x7f45d806ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.860+0000 7f45ddd84700 1 --2- 172.21.15.3:0/3028850912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d81601a0 0x7f45d806ab20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009e50 tx=0x7f45d000b330 comp rx=0 tx=0).stop 2023-11-04T05:28:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.861+0000 7f45ddd84700 1 -- 172.21.15.3:0/3028850912 shutdown_connections 2023-11-04T05:28:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.861+0000 7f45ddd84700 1 --2- 172.21.15.3:0/3028850912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d81601a0 0x7f45d806ab20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.861+0000 7f45ddd84700 1 --2- 172.21.15.3:0/3028850912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d80cc100 0x7f45d815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.861+0000 7f45ddd84700 1 -- 172.21.15.3:0/3028850912 >> 172.21.15.3:0/3028850912 conn(0x7f45d8072560 msgr2=0x7f45d8099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:17.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.861+0000 7f45ddd84700 1 -- 172.21.15.3:0/3028850912 shutdown_connections 2023-11-04T05:28:17.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.861+0000 7f45ddd84700 1 -- 172.21.15.3:0/3028850912 wait complete. 2023-11-04T05:28:17.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.862+0000 7f45ddd84700 1 Processor -- start 2023-11-04T05:28:17.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.862+0000 7f45ddd84700 1 -- start start 2023-11-04T05:28:17.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d80cc100 0x7f45d80c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d8155df0 0x7f45d80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 0x7f45d80c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d81f37e0 con 0x7f45d80ca130 2023-11-04T05:28:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d81f3aa0 con 0x7f45d80cc100 2023-11-04T05:28:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d81f3d80 con 0x7f45d8155df0 2023-11-04T05:28:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 0x7f45d80c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 0x7f45d80c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51376/0 (socket says 172.21.15.3:51376) 2023-11-04T05:28:17.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 -- 172.21.15.3:0/4194557363 learned_addr learned my addr 172.21.15.3:0/4194557363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d6ffd700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d8155df0 0x7f45d80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d77fe700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d80cc100 0x7f45d80c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 -- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d8155df0 msgr2=0x7f45d80c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d8155df0 0x7f45d80c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 -- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d80cc100 msgr2=0x7f45d80c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d80cc100 0x7f45d80c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 -- 172.21.15.3:0/4194557363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0009b30 con 0x7f45d80ca130 2023-11-04T05:28:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45d7fff700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 0x7f45d80c8d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45d80d1750 tx=0x7f45d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004800 con 0x7f45d80ca130 2023-11-04T05:28:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0004960 con 0x7f45d80ca130 2023-11-04T05:28:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d81f4060 con 0x7f45d80ca130 2023-11-04T05:28:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.863+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d001c680 con 0x7f45d80ca130 2023-11-04T05:28:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.864+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d81f4620 con 0x7f45d80ca130 2023-11-04T05:28:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.864+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d80b65a0 con 0x7f45d80ca130 2023-11-04T05:28:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.865+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45d001c7e0 con 0x7f45d80ca130 2023-11-04T05:28:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.865+0000 7f45cffff700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f45b806af00 0x7f45b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.865+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45d008aed0 con 0x7f45d80ca130 2023-11-04T05:28:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.865+0000 7f45d77fe700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f45b806af00 0x7f45b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.866+0000 7f45d77fe700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f45b806af00 0x7f45b806d3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007fd0 tx=0x7f45c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:17.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:17.870+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f45d00562c0 con 0x7f45d80ca130 2023-11-04T05:28:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.071+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45d80ce220 con 0x7f45d80ca130 2023-11-04T05:28:18.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.071+0000 7f45cffff700 1 -- 172.21.15.3:0/4194557363 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45d80ce220 con 0x7f45d80ca130 2023-11-04T05:28:18.074 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:28:18.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.073+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f45b806af00 msgr2=0x7f45b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.073+0000 7f45ddd84700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f45b806af00 0x7f45b806d3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007fd0 tx=0x7f45c0009040 comp rx=0 tx=0).stop 2023-11-04T05:28:18.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.073+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 msgr2=0x7f45d80c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.073+0000 7f45ddd84700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 0x7f45d80c8d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45d80d1750 tx=0x7f45d0004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 shutdown_connections 2023-11-04T05:28:18.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f45b806af00 0x7f45b806d3c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d80cc100 0x7f45d80c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45d80ca130 0x7f45d80c8d10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 --2- 172.21.15.3:0/4194557363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45d8155df0 0x7f45d80c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 >> 172.21.15.3:0/4194557363 conn(0x7f45d8072560 msgr2=0x7f45d815bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:18.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 shutdown_connections 2023-11-04T05:28:18.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.074+0000 7f45ddd84700 1 -- 172.21.15.3:0/4194557363 wait complete. 2023-11-04T05:28:18.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-04T05:28:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 Processor -- start 2023-11-04T05:28:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 -- start start 2023-11-04T05:28:18.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd188159310 0x7fd1880cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1880716c0 0x7fd18815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18815c280 con 0x7fd18806d790 2023-11-04T05:28:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18815bf80 con 0x7fd188159310 2023-11-04T05:28:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.210+0000 7fd18e321700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd188071eb0 con 0x7fd1880716c0 2023-11-04T05:28:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd188159310 0x7fd1880cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd188159310 0x7fd1880cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51380/0 (socket says 172.21.15.3:51380) 2023-11-04T05:28:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 -- 172.21.15.3:0/1613817040 learned_addr learned my addr 172.21.15.3:0/1613817040 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd1877fe700 1 --2- 172.21.15.3:0/1613817040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1880716c0 0x7fd18815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 -- 172.21.15.3:0/1613817040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1880716c0 msgr2=0x7fd18815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 --2- 172.21.15.3:0/1613817040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1880716c0 0x7fd18815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 -- 172.21.15.3:0/1613817040 >> v1:172.21.15.188:6789/0 conn(0x7fd18806d790 legacy=0x7fd18815f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 -- 172.21.15.3:0/1613817040 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188072190 con 0x7fd188159310 2023-11-04T05:28:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd187fff700 1 --2- 172.21.15.3:0/1613817040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd188159310 0x7fd1880cadb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1780098f0 tx=0x7fd178009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01f7436c19322df server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:18.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd186ffd700 1 -- 172.21.15.3:0/1613817040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800b8a0 con 0x7fd188159310 2023-11-04T05:28:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd186ffd700 1 -- 172.21.15.3:0/1613817040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17800ba00 con 0x7fd188159310 2023-11-04T05:28:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.211+0000 7fd186ffd700 1 -- 172.21.15.3:0/1613817040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800bd90 con 0x7fd188159310 2023-11-04T05:28:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 -- 172.21.15.3:0/1613817040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd188159310 msgr2=0x7fd1880cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 --2- 172.21.15.3:0/1613817040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd188159310 0x7fd1880cadb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1780098f0 tx=0x7fd178009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 -- 172.21.15.3:0/1613817040 shutdown_connections 2023-11-04T05:28:18.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 --2- 172.21.15.3:0/1613817040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd188159310 0x7fd1880cadb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 --2- 172.21.15.3:0/1613817040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1880716c0 0x7fd18815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 -- 172.21.15.3:0/1613817040 >> 172.21.15.3:0/1613817040 conn(0x7fd1880cfd50 msgr2=0x7fd1880d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:18.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 -- 172.21.15.3:0/1613817040 shutdown_connections 2023-11-04T05:28:18.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.212+0000 7fd18e321700 1 -- 172.21.15.3:0/1613817040 wait complete. 2023-11-04T05:28:18.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.213+0000 7fd18e321700 1 Processor -- start 2023-11-04T05:28:18.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.213+0000 7fd18e321700 1 -- start start 2023-11-04T05:28:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18e321700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18806d790 0x7fd188151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18e321700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 0x7fd1880c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18e321700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd188159310 0x7fd188156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18e321700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b39f0 con 0x7fd1880716c0 2023-11-04T05:28:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18e321700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880c7b50 con 0x7fd188159310 2023-11-04T05:28:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18e321700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b30e0 con 0x7fd18806d790 2023-11-04T05:28:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 0x7fd1880c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 0x7fd1880c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51388/0 (socket says 172.21.15.3:51388) 2023-11-04T05:28:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 -- 172.21.15.3:0/2445887231 learned_addr learned my addr 172.21.15.3:0/2445887231 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18c8be700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd188159310 0x7fd188156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd187fff700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18806d790 0x7fd188151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 -- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18806d790 msgr2=0x7fd188151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18806d790 0x7fd188151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 -- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd188159310 msgr2=0x7fd188156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd188159310 0x7fd188156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd1877fe700 1 -- 172.21.15.3:0/2445887231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd178009550 con 0x7fd1880716c0 2023-11-04T05:28:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd18c8be700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd188159310 0x7fd188156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.214+0000 7fd187fff700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18806d790 0x7fd188151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.215+0000 7fd1877fe700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 0x7fd1880c6f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e790 tx=0x7fd17c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:18.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.215+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005c20 con 0x7fd1880716c0 2023-11-04T05:28:18.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.215+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c005da0 con 0x7fd1880716c0 2023-11-04T05:28:18.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.215+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1880c7870 con 0x7fd1880716c0 2023-11-04T05:28:18.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.215+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c006260 con 0x7fd1880716c0 2023-11-04T05:28:18.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.215+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1880b3610 con 0x7fd1880716c0 2023-11-04T05:28:18.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.216+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188074df0 con 0x7fd1880716c0 2023-11-04T05:28:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.217+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd17c0063c0 con 0x7fd1880716c0 2023-11-04T05:28:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.217+0000 7fd184ff9700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd17406af30 0x7fd17406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.217+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd17c088b10 con 0x7fd1880716c0 2023-11-04T05:28:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.217+0000 7fd187fff700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd17406af30 0x7fd17406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.218+0000 7fd187fff700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd17406af30 0x7fd17406d3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd178014040 tx=0x7fd178008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:18.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.222+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd17c053fa0 con 0x7fd1880716c0 2023-11-04T05:28:18.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.425+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd18806e910 con 0x7fd1880716c0 2023-11-04T05:28:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.425+0000 7fd184ff9700 1 -- 172.21.15.3:0/2445887231 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd17c020020 con 0x7fd1880716c0 2023-11-04T05:28:18.428 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:18.428 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-11-04T05:28:18.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:28:18.429 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-11-04T05:28:18.429 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-11-04T05:28:18.429 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:28:18.429 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-11-04T05:28:18.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.427+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd17406af30 msgr2=0x7fd17406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.427+0000 7fd18e321700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd17406af30 0x7fd17406d3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd178014040 tx=0x7fd178008030 comp rx=0 tx=0).stop 2023-11-04T05:28:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.427+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 msgr2=0x7fd1880c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.427+0000 7fd18e321700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 0x7fd1880c6f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e790 tx=0x7fd17c00eb50 comp rx=0 tx=0).stop 2023-11-04T05:28:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.427+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 shutdown_connections 2023-11-04T05:28:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd17406af30 0x7fd17406d3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd188159310 0x7fd188156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18806d790 0x7fd188151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 --2- 172.21.15.3:0/2445887231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1880716c0 0x7fd1880c6f70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 >> 172.21.15.3:0/2445887231 conn(0x7fd1880cfd50 msgr2=0x7fd1881581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 shutdown_connections 2023-11-04T05:28:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.428+0000 7fd18e321700 1 -- 172.21.15.3:0/2445887231 wait complete. 2023-11-04T05:28:18.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-04T05:28:18.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-04T05:28:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.569+0000 7fe82e6d5700 1 Processor -- start 2023-11-04T05:28:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.569+0000 7fe82e6d5700 1 -- start start 2023-11-04T05:28:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe82e6d5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe82806ab20 0x7fe82806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe82e6d5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8280caca0 con 0x7fe8280d02d0 2023-11-04T05:28:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe82e6d5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8280cfe40 con 0x7fe8281611d0 2023-11-04T05:28:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe82e6d5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8280ce0a0 con 0x7fe82806ab20 2023-11-04T05:28:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe82806ab20 0x7fe82806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe82806ab20 0x7fe82806afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51396/0 (socket says 172.21.15.3:51396) 2023-11-04T05:28:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe82cc72700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe8280d02d0 0x7fe8280d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56744/0 (socket says 172.21.15.3:56744) 2023-11-04T05:28:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 -- 172.21.15.3:0/3376352561 learned_addr learned my addr 172.21.15.3:0/3376352561 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 -- 172.21.15.3:0/3376352561 >> v1:172.21.15.188:6789/0 conn(0x7fe8281611d0 legacy=0x7fe8280ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 -- 172.21.15.3:0/3376352561 >> v1:172.21.15.3:6789/0 conn(0x7fe8280d02d0 legacy=0x7fe8280d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 -- 172.21.15.3:0/3376352561 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280cf850 con 0x7fe82806ab20 2023-11-04T05:28:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.570+0000 7fe8277fe700 1 --2- 172.21.15.3:0/3376352561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe82806afa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8100098f0 tx=0x7fe810009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64f302a988d6a29 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe826ffd700 1 -- 172.21.15.3:0/3376352561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810004230 con 0x7fe82806ab20 2023-11-04T05:28:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe826ffd700 1 -- 172.21.15.3:0/3376352561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810004390 con 0x7fe82806ab20 2023-11-04T05:28:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe826ffd700 1 -- 172.21.15.3:0/3376352561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100036a0 con 0x7fe82806ab20 2023-11-04T05:28:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 -- 172.21.15.3:0/3376352561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 msgr2=0x7fe82806afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/3376352561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe82806afa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8100098f0 tx=0x7fe810009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 -- 172.21.15.3:0/3376352561 shutdown_connections 2023-11-04T05:28:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/3376352561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe82806afa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 -- 172.21.15.3:0/3376352561 >> 172.21.15.3:0/3376352561 conn(0x7fe828072560 msgr2=0x7fe828099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 -- 172.21.15.3:0/3376352561 shutdown_connections 2023-11-04T05:28:18.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.571+0000 7fe82e6d5700 1 -- 172.21.15.3:0/3376352561 wait complete. 2023-11-04T05:28:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.573+0000 7fe82e6d5700 1 Processor -- start 2023-11-04T05:28:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.573+0000 7fe82e6d5700 1 -- start start 2023-11-04T05:28:18.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82e6d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe828153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82e6d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8280d02d0 0x7fe8280b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82e6d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8281611d0 0x7fe8280c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82e6d5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280b4730 con 0x7fe82806ab20 2023-11-04T05:28:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82e6d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280b4bc0 con 0x7fe8280d02d0 2023-11-04T05:28:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82e6d5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280b3b20 con 0x7fe8281611d0 2023-11-04T05:28:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe828153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe828153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51400/0 (socket says 172.21.15.3:51400) 2023-11-04T05:28:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 -- 172.21.15.3:0/2405087347 learned_addr learned my addr 172.21.15.3:0/2405087347 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe8277fe700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8280d02d0 0x7fe8280b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82cc72700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8281611d0 0x7fe8280c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 -- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8281611d0 msgr2=0x7fe8280c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8281611d0 0x7fe8280c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 -- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8280d02d0 msgr2=0x7fe8280b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8280d02d0 0x7fe8280b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe827fff700 1 -- 172.21.15.3:0/2405087347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810009550 con 0x7fe82806ab20 2023-11-04T05:28:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe8277fe700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8280d02d0 0x7fe8280b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.574+0000 7fe82cc72700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8281611d0 0x7fe8280c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.575+0000 7fe827fff700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe828153b90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe81800b4a0 tx=0x7fe81800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.575+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818011830 con 0x7fe82806ab20 2023-11-04T05:28:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.575+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818011990 con 0x7fe82806ab20 2023-11-04T05:28:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.575+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8280b3e00 con 0x7fe82806ab20 2023-11-04T05:28:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.575+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818011d40 con 0x7fe82806ab20 2023-11-04T05:28:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.575+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8280b43c0 con 0x7fe82806ab20 2023-11-04T05:28:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.576+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8280b4a10 con 0x7fe82806ab20 2023-11-04T05:28:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.576+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe81801a400 con 0x7fe82806ab20 2023-11-04T05:28:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.577+0000 7fe824ff9700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe80806af00 0x7fe80806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.577+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe818088fd0 con 0x7fe82806ab20 2023-11-04T05:28:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.577+0000 7fe8277fe700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe80806af00 0x7fe80806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.577+0000 7fe8277fe700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe80806af00 0x7fe80806d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8280c7ef0 tx=0x7fe810008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:18.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.581+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fe818054480 con 0x7fe82806ab20 2023-11-04T05:28:18.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:18.781+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe8280b1920 con 0x7fe82806ab20 2023-11-04T05:28:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.421+0000 7fe824ff9700 1 -- 172.21.15.3:0/2405087347 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe818057aa0 con 0x7fe82806ab20 2023-11-04T05:28:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.425+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe80806af00 msgr2=0x7fe80806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.425+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe80806af00 0x7fe80806d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8280c7ef0 tx=0x7fe810008030 comp rx=0 tx=0).stop 2023-11-04T05:28:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 msgr2=0x7fe828153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe828153b90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe81800b4a0 tx=0x7fe81800b860 comp rx=0 tx=0).stop 2023-11-04T05:28:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 shutdown_connections 2023-11-04T05:28:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe80806af00 0x7fe80806d3c0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8280c7ef0 tx=0x7fe810008030 comp rx=0 tx=0).stop 2023-11-04T05:28:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8281611d0 0x7fe8280c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe82806ab20 0x7fe828153b90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 --2- 172.21.15.3:0/2405087347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8280d02d0 0x7fe8280b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 >> 172.21.15.3:0/2405087347 conn(0x7fe828072560 msgr2=0x7fe82806e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 shutdown_connections 2023-11-04T05:28:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.426+0000 7fe82e6d5700 1 -- 172.21.15.3:0/2405087347 wait complete. 2023-11-04T05:28:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-11-04T05:28:19.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-04T05:28:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.583+0000 7fb9988cb700 1 Processor -- start 2023-11-04T05:28:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.583+0000 7fb9988cb700 1 -- start start 2023-11-04T05:28:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb9988cb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb99415bca0 0x7fb9940cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb9988cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9940cf7c0 0x7fb99415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb9988cb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99415c360 con 0x7fb99415b8c0 2023-11-04T05:28:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb9988cb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99415c150 con 0x7fb9940cf7c0 2023-11-04T05:28:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb9988cb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99415c640 con 0x7fb99415bca0 2023-11-04T05:28:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb992d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9940cf7c0 0x7fb99415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb99259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb99415b8c0 0x7fb9940b7040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56756/0 (socket says 172.21.15.3:56756) 2023-11-04T05:28:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb99259c700 1 -- 172.21.15.3:0/1606663544 learned_addr learned my addr 172.21.15.3:0/1606663544 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb991d9b700 1 --2- 172.21.15.3:0/1606663544 >> v2:172.21.15.188:3300/0 conn(0x7fb99415bca0 0x7fb9940cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb99159a700 1 -- 172.21.15.3:0/1606663544 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590507467 0 0) 0x7fb99415c360 con 0x7fb99415b8c0 2023-11-04T05:28:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb99159a700 1 -- 172.21.15.3:0/1606663544 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb97c0035b0 con 0x7fb99415b8c0 2023-11-04T05:28:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb992d9d700 1 -- 172.21.15.3:0/1606663544 >> v2:172.21.15.188:3300/0 conn(0x7fb99415bca0 msgr2=0x7fb9940cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb992d9d700 1 --2- 172.21.15.3:0/1606663544 >> v2:172.21.15.188:3300/0 conn(0x7fb99415bca0 0x7fb9940cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb992d9d700 1 -- 172.21.15.3:0/1606663544 >> v1:172.21.15.3:6789/0 conn(0x7fb99415b8c0 legacy=0x7fb9940b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb992d9d700 1 -- 172.21.15.3:0/1606663544 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99415c920 con 0x7fb9940cf7c0 2023-11-04T05:28:19.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb992d9d700 1 --2- 172.21.15.3:0/1606663544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9940cf7c0 0x7fb99415f910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb9940b3d80 tx=0x7fb98c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c09736ab0cce568 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:19.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.585+0000 7fb99159a700 1 -- 172.21.15.3:0/1606663544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c00e070 con 0x7fb9940cf7c0 2023-11-04T05:28:19.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb99159a700 1 -- 172.21.15.3:0/1606663544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c00bb20 con 0x7fb9940cf7c0 2023-11-04T05:28:19.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb99159a700 1 -- 172.21.15.3:0/1606663544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c012430 con 0x7fb9940cf7c0 2023-11-04T05:28:19.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 -- 172.21.15.3:0/1606663544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9940cf7c0 msgr2=0x7fb99415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:19.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 --2- 172.21.15.3:0/1606663544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9940cf7c0 0x7fb99415f910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb9940b3d80 tx=0x7fb98c00b330 comp rx=0 tx=0).stop 2023-11-04T05:28:19.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 -- 172.21.15.3:0/1606663544 shutdown_connections 2023-11-04T05:28:19.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 --2- 172.21.15.3:0/1606663544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9940cf7c0 0x7fb99415f910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 --2- 172.21.15.3:0/1606663544 >> v2:172.21.15.188:3300/0 conn(0x7fb99415bca0 0x7fb9940cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 -- 172.21.15.3:0/1606663544 >> 172.21.15.3:0/1606663544 conn(0x7fb9940ce130 msgr2=0x7fb9940ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 -- 172.21.15.3:0/1606663544 shutdown_connections 2023-11-04T05:28:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.586+0000 7fb9988cb700 1 -- 172.21.15.3:0/1606663544 wait complete. 2023-11-04T05:28:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.588+0000 7fb9988cb700 1 Processor -- start 2023-11-04T05:28:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.588+0000 7fb9988cb700 1 -- start start 2023-11-04T05:28:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb9988cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9940cf7c0 0x7fb994150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb9988cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 0x7fb9940cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb9988cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99415bca0 0x7fb99414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:19.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb9988cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb994152120 con 0x7fb99415b8c0 2023-11-04T05:28:19.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb9988cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb994152400 con 0x7fb9940cf7c0 2023-11-04T05:28:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb9988cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9941526e0 con 0x7fb99415bca0 2023-11-04T05:28:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 0x7fb9940cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 0x7fb9940cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51416/0 (socket says 172.21.15.3:51416) 2023-11-04T05:28:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 -- 172.21.15.3:0/3197719368 learned_addr learned my addr 172.21.15.3:0/3197719368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb992d9d700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99415bca0 0x7fb99414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb99259c700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9940cf7c0 0x7fb994150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 -- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99415bca0 msgr2=0x7fb99414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99415bca0 0x7fb99414f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 -- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9940cf7c0 msgr2=0x7fb994150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9940cf7c0 0x7fb994150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 -- 172.21.15.3:0/3197719368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c009a70 con 0x7fb99415b8c0 2023-11-04T05:28:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb99259c700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9940cf7c0 0x7fb994150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.589+0000 7fb991d9b700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 0x7fb9940cb1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e870 tx=0x7fb98800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.590+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988009590 con 0x7fb99415b8c0 2023-11-04T05:28:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.590+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98800ee70 con 0x7fb99415b8c0 2023-11-04T05:28:19.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.590+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98800ca50 con 0x7fb99415b8c0 2023-11-04T05:28:19.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.590+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9941529c0 con 0x7fb99415b8c0 2023-11-04T05:28:19.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.590+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb994150960 con 0x7fb99415b8c0 2023-11-04T05:28:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.591+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9940ceeb0 con 0x7fb99415b8c0 2023-11-04T05:28:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.591+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb98800cbb0 con 0x7fb99415b8c0 2023-11-04T05:28:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.592+0000 7fb982ffd700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb97406af00 0x7fb97406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.592+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb988089af0 con 0x7fb99415b8c0 2023-11-04T05:28:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.592+0000 7fb99259c700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb97406af00 0x7fb97406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.593+0000 7fb99259c700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb97406af00 0x7fb97406d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb9840011e0 tx=0x7fb984009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.596+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fb988054e60 con 0x7fb99415b8c0 2023-11-04T05:28:19.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:19.796+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fb9940cc190 con 0x7fb99415b8c0 2023-11-04T05:28:20.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.428+0000 7fb982ffd700 1 -- 172.21.15.3:0/3197719368 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7fb9880170d0 con 0x7fb99415b8c0 2023-11-04T05:28:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb97406af00 msgr2=0x7fb97406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb97406af00 0x7fb97406d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb9840011e0 tx=0x7fb984009040 comp rx=0 tx=0).stop 2023-11-04T05:28:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 msgr2=0x7fb9940cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 0x7fb9940cb1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e870 tx=0x7fb98800ec30 comp rx=0 tx=0).stop 2023-11-04T05:28:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 shutdown_connections 2023-11-04T05:28:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb97406af00 0x7fb97406d3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9940cf7c0 0x7fb994150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99415bca0 0x7fb99414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 --2- 172.21.15.3:0/3197719368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb99415b8c0 0x7fb9940cb1a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.431+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 >> 172.21.15.3:0/3197719368 conn(0x7fb9940ce130 msgr2=0x7fb9940b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:20.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.432+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 shutdown_connections 2023-11-04T05:28:20.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.432+0000 7fb9988cb700 1 -- 172.21.15.3:0/3197719368 wait complete. 2023-11-04T05:28:20.436 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-11-04T05:28:20.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-04T05:28:20.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.593+0000 7fa943198700 1 Processor -- start 2023-11-04T05:28:20.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.593+0000 7fa943198700 1 -- start start 2023-11-04T05:28:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa943198700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa93c155df0 0x7fa93c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa943198700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa943198700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa93c1601a0 0x7fa93c06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa943198700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c0b0f20 con 0x7fa93c1601a0 2023-11-04T05:28:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa943198700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c15f4a0 con 0x7fa93c155df0 2023-11-04T05:28:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa943198700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c15c770 con 0x7fa93c15be50 2023-11-04T05:28:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa93c1601a0 0x7fa93c06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa93c1601a0 0x7fa93c06ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51426/0 (socket says 172.21.15.3:51426) 2023-11-04T05:28:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 -- 172.21.15.3:0/3635172910 learned_addr learned my addr 172.21.15.3:0/3635172910 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:20.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa940f34700 1 --2- 172.21.15.3:0/3635172910 >> v2:172.21.15.188:3300/0 conn(0x7fa93c155df0 0x7fa93c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa93bfff700 1 --2- 172.21.15.3:0/3635172910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 -- 172.21.15.3:0/3635172910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 msgr2=0x7fa93c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:20.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 --2- 172.21.15.3:0/3635172910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 -- 172.21.15.3:0/3635172910 >> v2:172.21.15.188:3300/0 conn(0x7fa93c155df0 msgr2=0x7fa93c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.594+0000 7fa941735700 1 --2- 172.21.15.3:0/3635172910 >> v2:172.21.15.188:3300/0 conn(0x7fa93c155df0 0x7fa93c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa941735700 1 -- 172.21.15.3:0/3635172910 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c15ca50 con 0x7fa93c1601a0 2023-11-04T05:28:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa941735700 1 --2- 172.21.15.3:0/3635172910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c1601a0 0x7fa93c06ab20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa93c09e710 tx=0x7fa93400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d9200d002a44df server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa93b7fe700 1 -- 172.21.15.3:0/3635172910 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400e070 con 0x7fa93c1601a0 2023-11-04T05:28:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa93b7fe700 1 -- 172.21.15.3:0/3635172910 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93400b9c0 con 0x7fa93c1601a0 2023-11-04T05:28:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa93b7fe700 1 -- 172.21.15.3:0/3635172910 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400bd70 con 0x7fa93c1601a0 2023-11-04T05:28:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 -- 172.21.15.3:0/3635172910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c1601a0 msgr2=0x7fa93c06ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 --2- 172.21.15.3:0/3635172910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c1601a0 0x7fa93c06ab20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa93c09e710 tx=0x7fa93400b330 comp rx=0 tx=0).stop 2023-11-04T05:28:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 -- 172.21.15.3:0/3635172910 shutdown_connections 2023-11-04T05:28:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 --2- 172.21.15.3:0/3635172910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c1601a0 0x7fa93c06ab20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 --2- 172.21.15.3:0/3635172910 >> v2:172.21.15.188:3300/0 conn(0x7fa93c155df0 0x7fa93c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 --2- 172.21.15.3:0/3635172910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 -- 172.21.15.3:0/3635172910 >> 172.21.15.3:0/3635172910 conn(0x7fa93c072560 msgr2=0x7fa93c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 -- 172.21.15.3:0/3635172910 shutdown_connections 2023-11-04T05:28:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.595+0000 7fa943198700 1 -- 172.21.15.3:0/3635172910 wait complete. 2023-11-04T05:28:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.597+0000 7fa943198700 1 Processor -- start 2023-11-04T05:28:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.597+0000 7fa943198700 1 -- start start 2023-11-04T05:28:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 0x7fa93c0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa93c152d10 0x7fa93c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0c9e30 con 0x7fa93c155df0 2023-11-04T05:28:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0ca060 con 0x7fa93c152d10 2023-11-04T05:28:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0ca340 con 0x7fa93c15be50 2023-11-04T05:28:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 0x7fa93c0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 0x7fa93c0cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51428/0 (socket says 172.21.15.3:51428) 2023-11-04T05:28:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 -- 172.21.15.3:0/2827660468 learned_addr learned my addr 172.21.15.3:0/2827660468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa93bfff700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa941735700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa93c152d10 0x7fa93c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 -- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 msgr2=0x7fa93c0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c0c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 -- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa93c152d10 msgr2=0x7fa93c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa93c152d10 0x7fa93c15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 -- 172.21.15.3:0/2827660468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934009b30 con 0x7fa93c155df0 2023-11-04T05:28:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa93bfff700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa940f34700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 0x7fa93c0cfdd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa93000e810 tx=0x7fa93000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930013610 con 0x7fa93c155df0 2023-11-04T05:28:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930013770 con 0x7fa93c155df0 2023-11-04T05:28:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c0ca620 con 0x7fa93c155df0 2023-11-04T05:28:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930013b20 con 0x7fa93c155df0 2023-11-04T05:28:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.598+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c0ae790 con 0x7fa93c155df0 2023-11-04T05:28:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.599+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c0aeb00 con 0x7fa93c155df0 2023-11-04T05:28:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.600+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa930013c80 con 0x7fa93c155df0 2023-11-04T05:28:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.600+0000 7fa9397fa700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa92006af00 0x7fa92006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.600+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa930089240 con 0x7fa93c155df0 2023-11-04T05:28:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.601+0000 7fa93bfff700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa92006af00 0x7fa92006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.601+0000 7fa93bfff700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa92006af00 0x7fa92006d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa93c15be00 tx=0x7fa928008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:20.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.605+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa930054360 con 0x7fa93c155df0 2023-11-04T05:28:20.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:20.851+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa93c074df0 con 0x7fa93c155df0 2023-11-04T05:28:21.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:28:21.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:28:21.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.429+0000 7fa9397fa700 1 -- 172.21.15.3:0/2827660468 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa930057980 con 0x7fa93c155df0 2023-11-04T05:28:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.433+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa92006af00 msgr2=0x7fa92006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.433+0000 7fa943198700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa92006af00 0x7fa92006d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa93c15be00 tx=0x7fa928008040 comp rx=0 tx=0).stop 2023-11-04T05:28:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.433+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 msgr2=0x7fa93c0cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.433+0000 7fa943198700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 0x7fa93c0cfdd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa93000e810 tx=0x7fa93000ebd0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 shutdown_connections 2023-11-04T05:28:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa92006af00 0x7fa92006d3c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa93c152d10 0x7fa93c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa93c155df0 0x7fa93c0cfdd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 --2- 172.21.15.3:0/2827660468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c15be50 0x7fa93c0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 >> 172.21.15.3:0/2827660468 conn(0x7fa93c072560 msgr2=0x7fa93c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:21.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.434+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 shutdown_connections 2023-11-04T05:28:21.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.435+0000 7fa943198700 1 -- 172.21.15.3:0/2827660468 wait complete. 2023-11-04T05:28:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2023-11-04T05:28:21.470 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-04T05:28:21.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-04T05:28:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.643+0000 7f60fedd1700 1 Processor -- start 2023-11-04T05:28:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.643+0000 7f60fedd1700 1 -- start start 2023-11-04T05:28:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60fedd1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60f80cc100 0x7f60f815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60fedd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f81601a0 0x7f60f806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60fedd1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f80b0f20 con 0x7f60f8155df0 2023-11-04T05:28:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60fedd1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f80cc590 con 0x7f60f80cc100 2023-11-04T05:28:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60fedd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f815c560 con 0x7f60f81601a0 2023-11-04T05:28:21.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60f7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60f80cc100 0x7f60f815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:21.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60f7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60f80cc100 0x7f60f815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51438/0 (socket says 172.21.15.3:51438) 2023-11-04T05:28:21.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60f7fff700 1 -- 172.21.15.3:0/2644173688 learned_addr learned my addr 172.21.15.3:0/2644173688 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:21.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.644+0000 7f60fcb6d700 1 --1- 172.21.15.3:0/2644173688 >> v1:172.21.15.3:6789/0 conn(0x7f60f8155df0 0x7f60f815be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56784/0 (socket says 172.21.15.3:56784) 2023-11-04T05:28:21.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60fd36e700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f81601a0 0x7f60f806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:21.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f7fff700 1 -- 172.21.15.3:0/2644173688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f81601a0 msgr2=0x7f60f806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:21.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f7fff700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f81601a0 0x7f60f806ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f7fff700 1 -- 172.21.15.3:0/2644173688 >> v1:172.21.15.3:6789/0 conn(0x7f60f8155df0 legacy=0x7f60f815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:21.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f7fff700 1 -- 172.21.15.3:0/2644173688 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f815f4a0 con 0x7f60f80cc100 2023-11-04T05:28:21.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60fd36e700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f81601a0 0x7f60f806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:21.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f7fff700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f80cc100 0x7f60f815fc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60e40098f0 tx=0x7f60e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdda9e364065eda server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:21.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f77fe700 1 -- 172.21.15.3:0/2644173688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e400b8a0 con 0x7f60f80cc100 2023-11-04T05:28:21.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f77fe700 1 -- 172.21.15.3:0/2644173688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e400ba00 con 0x7f60f80cc100 2023-11-04T05:28:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.645+0000 7f60f77fe700 1 -- 172.21.15.3:0/2644173688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e400bd90 con 0x7f60f80cc100 2023-11-04T05:28:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 -- 172.21.15.3:0/2644173688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f80cc100 msgr2=0x7f60f815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f80cc100 0x7f60f815fc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60e40098f0 tx=0x7f60e4009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 -- 172.21.15.3:0/2644173688 shutdown_connections 2023-11-04T05:28:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f81601a0 0x7f60f806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 --2- 172.21.15.3:0/2644173688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f80cc100 0x7f60f815fc60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 -- 172.21.15.3:0/2644173688 >> 172.21.15.3:0/2644173688 conn(0x7f60f8072560 msgr2=0x7f60f8099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 -- 172.21.15.3:0/2644173688 shutdown_connections 2023-11-04T05:28:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.646+0000 7f60fedd1700 1 -- 172.21.15.3:0/2644173688 wait complete. 2023-11-04T05:28:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.647+0000 7f60fedd1700 1 Processor -- start 2023-11-04T05:28:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.647+0000 7f60fedd1700 1 -- start start 2023-11-04T05:28:21.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.648+0000 7f60fedd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80cc100 0x7f60f80c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:21.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.648+0000 7f60fedd1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 0x7f60f806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:21.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.648+0000 7f60fedd1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f806eb90 0x7f60f80c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:21.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.648+0000 7f60fedd1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f806f010 con 0x7f60f8155df0 2023-11-04T05:28:21.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.648+0000 7f60fedd1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80c6830 con 0x7f60f806eb90 2023-11-04T05:28:21.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.648+0000 7f60fedd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80c6b10 con 0x7f60f80cc100 2023-11-04T05:28:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 0x7f60f806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 0x7f60f806e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51444/0 (socket says 172.21.15.3:51444) 2023-11-04T05:28:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 -- 172.21.15.3:0/320427586 learned_addr learned my addr 172.21.15.3:0/320427586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60fcb6d700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80cc100 0x7f60f80c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60fd36e700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f806eb90 0x7f60f80c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 -- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80cc100 msgr2=0x7f60f80c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80cc100 0x7f60f80c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 -- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f806eb90 msgr2=0x7f60f80c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f806eb90 0x7f60f80c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 -- 172.21.15.3:0/320427586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4009550 con 0x7f60f8155df0 2023-11-04T05:28:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f7fff700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 0x7f60f806e650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60e4000c00 tx=0x7f60e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:21.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4004380 con 0x7f60f8155df0 2023-11-04T05:28:21.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e401f070 con 0x7f60f8155df0 2023-11-04T05:28:21.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f80c6df0 con 0x7f60f8155df0 2023-11-04T05:28:21.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40036a0 con 0x7f60f8155df0 2023-11-04T05:28:21.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.649+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f80c3730 con 0x7f60f8155df0 2023-11-04T05:28:21.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.650+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f80c38c0 con 0x7f60f8155df0 2023-11-04T05:28:21.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.651+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60e4021030 con 0x7f60f8155df0 2023-11-04T05:28:21.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.651+0000 7f60f57fa700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f60dc06af00 0x7f60dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:21.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.651+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60e408a960 con 0x7f60f8155df0 2023-11-04T05:28:21.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.651+0000 7f60fcb6d700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f60dc06af00 0x7f60dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:21.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.652+0000 7f60fcb6d700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f60dc06af00 0x7f60dc06d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f60 tx=0x7f60ec007ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:21.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.656+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f60e4055a20 con 0x7f60f8155df0 2023-11-04T05:28:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:21.868+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f60f80c6fe0 con 0x7f60f8155df0 2023-11-04T05:28:22.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.446+0000 7f60f57fa700 1 -- 172.21.15.3:0/320427586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7f60e4059040 con 0x7f60f8155df0 2023-11-04T05:28:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.449+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f60dc06af00 msgr2=0x7f60dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.449+0000 7f60fedd1700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f60dc06af00 0x7f60dc06d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f60 tx=0x7f60ec007ed0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.449+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 msgr2=0x7f60f806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.449+0000 7f60fedd1700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 0x7f60f806e650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60e4000c00 tx=0x7f60e4004750 comp rx=0 tx=0).stop 2023-11-04T05:28:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.449+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 shutdown_connections 2023-11-04T05:28:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f60dc06af00 0x7f60dc06d3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f806eb90 0x7f60f80c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80cc100 0x7f60f80c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 --2- 172.21.15.3:0/320427586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60f8155df0 0x7f60f806e650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 >> 172.21.15.3:0/320427586 conn(0x7f60f8072560 msgr2=0x7f60f806ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 shutdown_connections 2023-11-04T05:28:22.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.450+0000 7f60fedd1700 1 -- 172.21.15.3:0/320427586 wait complete. 2023-11-04T05:28:22.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-04T05:28:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.605+0000 7f4c33735700 1 Processor -- start 2023-11-04T05:28:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.605+0000 7f4c33735700 1 -- start start 2023-11-04T05:28:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c33735700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15de80 0x7f4c2c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c33735700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c0cf5a0 con 0x7f4c2c072710 2023-11-04T05:28:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c33735700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c0cb060 con 0x7f4c2c0ce460 2023-11-04T05:28:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c33735700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c0cea20 con 0x7f4c2c15de80 2023-11-04T05:28:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c31cd2700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4c2c072710 0x7f4c2c0cb500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56800/0 (socket says 172.21.15.3:56800) 2023-11-04T05:28:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c31cd2700 1 -- 172.21.15.3:0/1600956532 learned_addr learned my addr 172.21.15.3:0/1600956532 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999082901 0 0) 0x7f4c24001490 con 0x7f4c2c072710 2023-11-04T05:28:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c314d1700 1 --2- 172.21.15.3:0/1600956532 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15de80 0x7f4c2c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c180035b0 con 0x7f4c2c072710 2023-11-04T05:28:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522786050 0 0) 0x7f4c180035b0 con 0x7f4c2c072710 2023-11-04T05:28:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15de80 msgr2=0x7f4c2c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 --2- 172.21.15.3:0/1600956532 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15de80 0x7f4c2c15e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 >> v1:172.21.15.188:6789/0 conn(0x7f4c2c0ce460 legacy=0x7f4c2c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0ced00 con 0x7f4c2c072710 2023-11-04T05:28:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f4c24001b80 con 0x7f4c2c072710 2023-11-04T05:28:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 >> v1:172.21.15.3:6789/0 conn(0x7f4c2c072710 legacy=0x7f4c2c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c18003a00 0x7f4c18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18004850 0x7f4c18004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c18005210 0x7f4c1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0cea20 con 0x7f4c18005210 2023-11-04T05:28:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1800c1b0 con 0x7f4c18003a00 2023-11-04T05:28:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.606+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1800c370 con 0x7f4c18004850 2023-11-04T05:28:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c18005210 0x7f4c1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c314d1700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c18003a00 0x7f4c18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c30cd0700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18004850 0x7f4c18004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 -- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18004850 msgr2=0x7f4c18004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18004850 0x7f4c18004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 -- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c18003a00 msgr2=0x7f4c18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:22.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c18003a00 0x7f4c18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 -- 172.21.15.3:0/1600956532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c180035b0 con 0x7f4c18005210 2023-11-04T05:28:22.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 -- 172.21.15.3:0/1600956532 reap_dead start 2023-11-04T05:28:22.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c31cd2700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c18005210 0x7f4c1800ba10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0c4120 tx=0x7f4c18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d10145e4128bc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:22.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400a490 con 0x7f4c18005210 2023-11-04T05:28:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24012620 con 0x7f4c18005210 2023-11-04T05:28:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.607+0000 7f4c23fff700 1 -- 172.21.15.3:0/1600956532 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240129d0 con 0x7f4c18005210 2023-11-04T05:28:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 -- 172.21.15.3:0/1600956532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c18005210 msgr2=0x7f4c1800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c18005210 0x7f4c1800ba10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0c4120 tx=0x7f4c18005880 comp rx=0 tx=0).stop 2023-11-04T05:28:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 -- 172.21.15.3:0/1600956532 shutdown_connections 2023-11-04T05:28:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 --2- 172.21.15.3:0/1600956532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c18005210 0x7f4c1800ba10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 -- 172.21.15.3:0/1600956532 >> 172.21.15.3:0/1600956532 conn(0x7f4c2c0d0680 msgr2=0x7f4c2c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 -- 172.21.15.3:0/1600956532 shutdown_connections 2023-11-04T05:28:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.608+0000 7f4c33735700 1 -- 172.21.15.3:0/1600956532 wait complete. 2023-11-04T05:28:22.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.609+0000 7f4c33735700 1 Processor -- start 2023-11-04T05:28:22.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.609+0000 7f4c33735700 1 -- start start 2023-11-04T05:28:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c33735700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18005210 0x7f4c2c0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c33735700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 0x7f4c2c0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c33735700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c14b730 0x7f4c2c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c33735700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c14c0f0 con 0x7f4c2c15cac0 2023-11-04T05:28:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c33735700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0aef80 con 0x7f4c2c14b730 2023-11-04T05:28:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c33735700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0af260 con 0x7f4c18005210 2023-11-04T05:28:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.610+0000 7f4c30cd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 0x7f4c2c0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 0x7f4c2c0ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51462/0 (socket says 172.21.15.3:51462) 2023-11-04T05:28:22.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 -- 172.21.15.3:0/3470305063 learned_addr learned my addr 172.21.15.3:0/3470305063 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:22.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c314d1700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18005210 0x7f4c2c0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c31cd2700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c14b730 0x7f4c2c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 -- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18005210 msgr2=0x7f4c2c0aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18005210 0x7f4c2c0aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 -- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c14b730 msgr2=0x7f4c2c14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:22.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c14b730 0x7f4c2c14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:22.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 -- 172.21.15.3:0/3470305063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0af540 con 0x7f4c2c15cac0 2023-11-04T05:28:22.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c314d1700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18005210 0x7f4c2c0aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:22.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c30cd0700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 0x7f4c2c0ae620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00f820 tx=0x7f4c1c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:22.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0052e0 con 0x7f4c2c15cac0 2023-11-04T05:28:22.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c005460 con 0x7f4c2c15cac0 2023-11-04T05:28:22.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c0af820 con 0x7f4c2c15cac0 2023-11-04T05:28:22.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0038c0 con 0x7f4c2c15cac0 2023-11-04T05:28:22.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.611+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c15cf40 con 0x7f4c2c15cac0 2023-11-04T05:28:22.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.612+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c15d0d0 con 0x7f4c2c15cac0 2023-11-04T05:28:22.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.612+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c1c003a20 con 0x7f4c2c15cac0 2023-11-04T05:28:22.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.613+0000 7f4c21ffb700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c1006af00 0x7f4c1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:22.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.613+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f4c1c089470 con 0x7f4c2c15cac0 2023-11-04T05:28:22.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.613+0000 7f4c314d1700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c1006af00 0x7f4c1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:22.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.613+0000 7f4c314d1700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c1006af00 0x7f4c1006d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009be0 tx=0x7f4c28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:22.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.617+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4c2c15d0d0 con 0x7f4c2c15cac0 2023-11-04T05:28:22.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:22.817+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4c2c15d0d0 con 0x7f4c2c15cac0 2023-11-04T05:28:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.824+0000 7f4c21ffb700 1 -- 172.21.15.3:0/3470305063 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7f4c2c15d0d0 con 0x7f4c2c15cac0 2023-11-04T05:28:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c1006af00 msgr2=0x7f4c1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c1006af00 0x7f4c1006d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009be0 tx=0x7f4c28008040 comp rx=0 tx=0).stop 2023-11-04T05:28:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 msgr2=0x7f4c2c0ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 0x7f4c2c0ae620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00f820 tx=0x7f4c1c00fbe0 comp rx=0 tx=0).stop 2023-11-04T05:28:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 shutdown_connections 2023-11-04T05:28:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c1006af00 0x7f4c1006d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c14b730 0x7f4c2c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c18005210 0x7f4c2c0aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 --2- 172.21.15.3:0/3470305063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c15cac0 0x7f4c2c0ae620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 >> 172.21.15.3:0/3470305063 conn(0x7f4c2c0d0680 msgr2=0x7f4c2c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 shutdown_connections 2023-11-04T05:28:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.827+0000 7f4c33735700 1 -- 172.21.15.3:0/3470305063 wait complete. 2023-11-04T05:28:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' created 2023-11-04T05:28:24.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-04T05:28:24.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.984+0000 7f5ebd99d700 1 Processor -- start 2023-11-04T05:28:24.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.984+0000 7f5ebd99d700 1 -- start start 2023-11-04T05:28:24.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd99d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5eb806f2f0 0x7f5eb806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:24.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd99d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb806e300 con 0x7f5eb80ce930 2023-11-04T05:28:24.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd99d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80cbca0 con 0x7f5eb8159fd0 2023-11-04T05:28:24.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd99d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb806e530 con 0x7f5eb806f2f0 2023-11-04T05:28:24.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd19c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5eb806f2f0 0x7f5eb806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:24.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebc99b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5eb80ce930 0x7f5eb80ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56816/0 (socket says 172.21.15.3:56816) 2023-11-04T05:28:24.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd19c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5eb806f2f0 0x7f5eb806f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51472/0 (socket says 172.21.15.3:51472) 2023-11-04T05:28:24.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebc99b700 1 -- 172.21.15.3:0/1101110087 learned_addr learned my addr 172.21.15.3:0/1101110087 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:24.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd19c700 1 -- 172.21.15.3:0/1101110087 >> v1:172.21.15.188:6789/0 conn(0x7f5eb8159fd0 legacy=0x7f5eb80cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:24.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd19c700 1 -- 172.21.15.3:0/1101110087 >> v1:172.21.15.3:6789/0 conn(0x7f5eb80ce930 legacy=0x7f5eb80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:24.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5eb77fe700 1 -- 172.21.15.3:0/1101110087 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578629619 0 0) 0x7f5eb806e300 con 0x7f5eb80ce930 2023-11-04T05:28:24.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd19c700 1 -- 172.21.15.3:0/1101110087 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb806e810 con 0x7f5eb806f2f0 2023-11-04T05:28:24.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5ebd19c700 1 --2- 172.21.15.3:0/1101110087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb806f780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000aea0 tx=0x7f5eb000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b36de05719655f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:24.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5eb77fe700 1 -- 172.21.15.3:0/1101110087 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003c20 con 0x7f5eb806f2f0 2023-11-04T05:28:24.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5eb77fe700 1 -- 172.21.15.3:0/1101110087 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0003d80 con 0x7f5eb806f2f0 2023-11-04T05:28:24.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.985+0000 7f5eb77fe700 1 -- 172.21.15.3:0/1101110087 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000cbc0 con 0x7f5eb806f2f0 2023-11-04T05:28:24.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 -- 172.21.15.3:0/1101110087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 msgr2=0x7f5eb806f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:24.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/1101110087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb806f780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000aea0 tx=0x7f5eb000c330 comp rx=0 tx=0).stop 2023-11-04T05:28:24.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 -- 172.21.15.3:0/1101110087 shutdown_connections 2023-11-04T05:28:24.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/1101110087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb806f780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:24.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 -- 172.21.15.3:0/1101110087 >> 172.21.15.3:0/1101110087 conn(0x7f5eb80cc580 msgr2=0x7f5eb80cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:24.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 -- 172.21.15.3:0/1101110087 shutdown_connections 2023-11-04T05:28:24.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.986+0000 7f5ebd99d700 1 -- 172.21.15.3:0/1101110087 wait complete. 2023-11-04T05:28:24.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.987+0000 7f5ebd99d700 1 Processor -- start 2023-11-04T05:28:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.987+0000 7f5ebd99d700 1 -- start start 2023-11-04T05:28:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd99d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb8159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd99d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80ce930 0x7f5eb80ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd99d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb8159fd0 0x7f5eb806b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd99d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80ad830 con 0x7f5eb806f2f0 2023-11-04T05:28:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd99d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81f3390 con 0x7f5eb8159fd0 2023-11-04T05:28:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd99d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81f3670 con 0x7f5eb80ce930 2023-11-04T05:28:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebc99b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb8159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebc99b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb8159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51474/0 (socket says 172.21.15.3:51474) 2023-11-04T05:28:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebc99b700 1 -- 172.21.15.3:0/3850481349 learned_addr learned my addr 172.21.15.3:0/3850481349 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5eb7fff700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80ce930 0x7f5eb80ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebd19c700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb8159fd0 0x7f5eb806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.988+0000 7f5ebc99b700 1 -- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80ce930 msgr2=0x7f5eb80ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebc99b700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80ce930 0x7f5eb80ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebc99b700 1 -- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb8159fd0 msgr2=0x7f5eb806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebc99b700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb8159fd0 0x7f5eb806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebc99b700 1 -- 172.21.15.3:0/3850481349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb000ab80 con 0x7f5eb806f2f0 2023-11-04T05:28:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5eb7fff700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80ce930 0x7f5eb80ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebd19c700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb8159fd0 0x7f5eb806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebc99b700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb8159030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0011e0 tx=0x7f5eac00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac004c80 con 0x7f5eb806f2f0 2023-11-04T05:28:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac004e00 con 0x7f5eb806f2f0 2023-11-04T05:28:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb81f3950 con 0x7f5eb806f2f0 2023-11-04T05:28:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac007310 con 0x7f5eb806f2f0 2023-11-04T05:28:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.989+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb81f3f10 con 0x7f5eb806f2f0 2023-11-04T05:28:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.990+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb804f730 con 0x7f5eb806f2f0 2023-11-04T05:28:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.990+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5eac007470 con 0x7f5eb806f2f0 2023-11-04T05:28:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.991+0000 7f5eb57fa700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:25.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.991+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f5eac0896c0 con 0x7f5eb806f2f0 2023-11-04T05:28:25.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.991+0000 7f5eb7fff700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:25.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.991+0000 7f5eb7fff700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b1480 tx=0x7f5ea4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:25.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:24.996+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5eac0544a0 con 0x7f5eb806f2f0 2023-11-04T05:28:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.198+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f5eb815c4a0 con 0x7f5eb806f2f0 2023-11-04T05:28:25.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.829+0000 7f5eb57fa700 1 -- 172.21.15.3:0/3850481349 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5eac057ac0 con 0x7f5eb806f2f0 2023-11-04T05:28:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.834+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 msgr2=0x7f5e9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.834+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b1480 tx=0x7f5ea4009040 comp rx=0 tx=0).stop 2023-11-04T05:28:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.834+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 msgr2=0x7f5eb8159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:25.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.834+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb8159030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0011e0 tx=0x7f5eac00cae0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 shutdown_connections 2023-11-04T05:28:25.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb8159fd0 0x7f5eb806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb806f2f0 0x7f5eb8159030 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 --2- 172.21.15.3:0/3850481349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80ce930 0x7f5eb80ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 >> 172.21.15.3:0/3850481349 conn(0x7f5eb80cc580 msgr2=0x7f5eb80af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:25.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 shutdown_connections 2023-11-04T05:28:25.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:25.835+0000 7f5ebd99d700 1 -- 172.21.15.3:0/3850481349 wait complete. 2023-11-04T05:28:25.848 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 4 allow_ec_overwrites to true 2023-11-04T05:28:25.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-04T05:28:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.002+0000 7fc5d66a6700 1 Processor -- start 2023-11-04T05:28:26.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.002+0000 7fc5d66a6700 1 -- start start 2023-11-04T05:28:26.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.002+0000 7fc5d66a6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5d00ceaa0 0x7fc5d00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.002+0000 7fc5d66a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d015b870 con 0x7fc5d015b430 2023-11-04T05:28:26.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5d66a6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d00ccc70 con 0x7fc5d00cee80 2023-11-04T05:28:26.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5d66a6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00cdee0 con 0x7fc5d00ceaa0 2023-11-04T05:28:26.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5d4c43700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc5d015b430 0x7fc5d00caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56834/0 (socket says 172.21.15.3:56834) 2023-11-04T05:28:26.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5d4c43700 1 -- 172.21.15.3:0/4119529688 learned_addr learned my addr 172.21.15.3:0/4119529688 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:26.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5cffff700 1 --2- 172.21.15.3:0/4119529688 >> v2:172.21.15.188:3300/0 conn(0x7fc5d00ceaa0 0x7fc5d00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623213114 0 0) 0x7fc5d015b870 con 0x7fc5d015b430 2023-11-04T05:28:26.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b80035b0 con 0x7fc5d015b430 2023-11-04T05:28:26.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177277287 0 0) 0x7fc5b80035b0 con 0x7fc5d015b430 2023-11-04T05:28:26.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 >> v2:172.21.15.188:3300/0 conn(0x7fc5d00ceaa0 msgr2=0x7fc5d00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:26.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 --2- 172.21.15.3:0/4119529688 >> v2:172.21.15.188:3300/0 conn(0x7fc5d00ceaa0 0x7fc5d00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 >> v1:172.21.15.188:6789/0 conn(0x7fc5d00cee80 legacy=0x7fc5d015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d015baa0 con 0x7fc5d015b430 2023-11-04T05:28:26.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fc5c80019f0 con 0x7fc5d015b430 2023-11-04T05:28:26.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 >> v1:172.21.15.3:6789/0 conn(0x7fc5d015b430 legacy=0x7fc5d00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b8003a00 0x7fc5b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b8004850 0x7fc5b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8005210 0x7fc5b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00cdee0 con 0x7fc5b8003a00 2023-11-04T05:28:26.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b800c1b0 con 0x7fc5b8005210 2023-11-04T05:28:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b800c370 con 0x7fc5b8004850 2023-11-04T05:28:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5cffff700 1 --2- 172.21.15.3:0/4119529688 >> v2:172.21.15.188:3300/0 conn(0x7fc5d00ceaa0 0x7fc5d00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5d4c43700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8005210 0x7fc5b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5cffff700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b8003a00 0x7fc5b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.003+0000 7fc5cf7fe700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b8004850 0x7fc5b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5cffff700 1 -- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b8004850 msgr2=0x7fc5b8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5cffff700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b8004850 0x7fc5b8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5cffff700 1 -- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8005210 msgr2=0x7fc5b800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5cffff700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8005210 0x7fc5b800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5d4c43700 1 -- 172.21.15.3:0/4119529688 reap_dead start 2023-11-04T05:28:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5cffff700 1 -- 172.21.15.3:0/4119529688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80035b0 con 0x7fc5b8003a00 2023-11-04T05:28:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5cffff700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b8003a00 0x7fc5b8003e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009d10 tx=0x7fc5b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43899d79f15dfd7e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0009e60 con 0x7fc5b8003a00 2023-11-04T05:28:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c000e9a0 con 0x7fc5b8003a00 2023-11-04T05:28:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.004+0000 7fc5ceffd700 1 -- 172.21.15.3:0/4119529688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000ed30 con 0x7fc5b8003a00 2023-11-04T05:28:26.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 -- 172.21.15.3:0/4119529688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b8003a00 msgr2=0x7fc5b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b8003a00 0x7fc5b8003e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009d10 tx=0x7fc5b8005880 comp rx=0 tx=0).stop 2023-11-04T05:28:26.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 -- 172.21.15.3:0/4119529688 shutdown_connections 2023-11-04T05:28:26.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/4119529688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b8003a00 0x7fc5b8003e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 -- 172.21.15.3:0/4119529688 >> 172.21.15.3:0/4119529688 conn(0x7fc5d0072560 msgr2=0x7fc5d0099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:26.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 -- 172.21.15.3:0/4119529688 shutdown_connections 2023-11-04T05:28:26.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.005+0000 7fc5d66a6700 1 -- 172.21.15.3:0/4119529688 wait complete. 2023-11-04T05:28:26.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.006+0000 7fc5d66a6700 1 Processor -- start 2023-11-04T05:28:26.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.006+0000 7fc5d66a6700 1 -- start start 2023-11-04T05:28:26.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d66a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8003a00 0x7fc5d01f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d66a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 0x7fc5d01f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d66a6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d00b3cf0 0x7fc5d006e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d66a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01f7320 con 0x7fc5d00ae680 2023-11-04T05:28:26.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d66a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01f6d60 con 0x7fc5b8003a00 2023-11-04T05:28:26.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d66a6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01f7040 con 0x7fc5d00b3cf0 2023-11-04T05:28:26.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 0x7fc5d01f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 0x7fc5d01f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51496/0 (socket says 172.21.15.3:51496) 2023-11-04T05:28:26.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 -- 172.21.15.3:0/1941341219 learned_addr learned my addr 172.21.15.3:0/1941341219 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:26.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5d4c43700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d00b3cf0 0x7fc5d006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cffff700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8003a00 0x7fc5d01f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 -- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d00b3cf0 msgr2=0x7fc5d006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d00b3cf0 0x7fc5d006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 -- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8003a00 msgr2=0x7fc5d01f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8003a00 0x7fc5d01f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 -- 172.21.15.3:0/1941341219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00c77c0 con 0x7fc5d00ae680 2023-11-04T05:28:26.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5cf7fe700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 0x7fc5d01f6480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400f7f0 tx=0x7fc5c400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:26.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.007+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40052e0 con 0x7fc5d00ae680 2023-11-04T05:28:26.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.008+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4005440 con 0x7fc5d00ae680 2023-11-04T05:28:26.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.008+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d00c7aa0 con 0x7fc5d00ae680 2023-11-04T05:28:26.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.008+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40038a0 con 0x7fc5d00ae680 2023-11-04T05:28:26.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.008+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d01f69c0 con 0x7fc5d00ae680 2023-11-04T05:28:26.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.008+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d00c7ce0 con 0x7fc5d00ae680 2023-11-04T05:28:26.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.009+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5c4003a00 con 0x7fc5d00ae680 2023-11-04T05:28:26.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.009+0000 7fc5ccff9700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5bc06af30 0x7fc5bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.009+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fc5c40896d0 con 0x7fc5d00ae680 2023-11-04T05:28:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.009+0000 7fc5cffff700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5bc06af30 0x7fc5bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.010+0000 7fc5cffff700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5bc06af30 0x7fc5bc06d3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00090b0 tx=0x7fc5c0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:26.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.014+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fc5d00c7ce0 con 0x7fc5d00ae680 2023-11-04T05:28:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.263+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fc5d00c7ce0 con 0x7fc5d00ae680 2023-11-04T05:28:26.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.831+0000 7fc5ccff9700 1 -- 172.21.15.3:0/1941341219 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fc5d00c7ce0 con 0x7fc5d00ae680 2023-11-04T05:28:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5bc06af30 msgr2=0x7fc5bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5bc06af30 0x7fc5bc06d3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00090b0 tx=0x7fc5c0008030 comp rx=0 tx=0).stop 2023-11-04T05:28:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 msgr2=0x7fc5d01f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 0x7fc5d01f6480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400f7f0 tx=0x7fc5c400fbb0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 shutdown_connections 2023-11-04T05:28:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5bc06af30 0x7fc5bc06d3f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8003a00 0x7fc5d01f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d00b3cf0 0x7fc5d006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.833+0000 7fc5d66a6700 1 --2- 172.21.15.3:0/1941341219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5d00ae680 0x7fc5d01f6480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.834+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 >> 172.21.15.3:0/1941341219 conn(0x7fc5d0072560 msgr2=0x7fc5d00736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:26.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.834+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 shutdown_connections 2023-11-04T05:28:26.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.834+0000 7fc5d66a6700 1 -- 172.21.15.3:0/1941341219 wait complete. 2023-11-04T05:28:26.839 INFO:teuthology.orchestra.run.smithi003.stderr:added data pool 4 to fsmap 2023-11-04T05:28:26.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:26.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.980+0000 7f66b359e700 1 Processor -- start 2023-11-04T05:28:26.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.980+0000 7f66b359e700 1 -- start start 2023-11-04T05:28:26.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b359e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66b415b9c0 0x7f66b41601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b359e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66b40cd440 0x7f66b40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40cdda0 con 0x7f66b40cd440 2023-11-04T05:28:26.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b359e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40cb6d0 con 0x7f66b415b9c0 2023-11-04T05:28:26.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b359e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40cb9b0 con 0x7f66b40b6260 2023-11-04T05:28:26.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66b40cd440 0x7f66b40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b2d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66b40cd440 0x7f66b40cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51504/0 (socket says 172.21.15.3:51504) 2023-11-04T05:28:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.981+0000 7f66b2d9d700 1 -- 172.21.15.3:0/3773792033 learned_addr learned my addr 172.21.15.3:0/3773792033 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b1d9b700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b259c700 1 --2- 172.21.15.3:0/3773792033 >> v2:172.21.15.188:3300/0 conn(0x7f66b415b9c0 0x7f66b41601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b2d9d700 1 -- 172.21.15.3:0/3773792033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 msgr2=0x7f66b40cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:26.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b2d9d700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b2d9d700 1 -- 172.21.15.3:0/3773792033 >> v2:172.21.15.188:3300/0 conn(0x7f66b415b9c0 msgr2=0x7f66b41601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:26.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b2d9d700 1 --2- 172.21.15.3:0/3773792033 >> v2:172.21.15.188:3300/0 conn(0x7f66b415b9c0 0x7f66b41601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b2d9d700 1 -- 172.21.15.3:0/3773792033 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b415f6e0 con 0x7f66b40cd440 2023-11-04T05:28:26.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b1d9b700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:26.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b259c700 1 --2- 172.21.15.3:0/3773792033 >> v2:172.21.15.188:3300/0 conn(0x7f66b415b9c0 0x7f66b41601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:26.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b2d9d700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b40cd440 0x7f66b40cd860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66b406b7e0 tx=0x7f66ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24df66c2fa5e76d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:26.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b159a700 1 -- 172.21.15.3:0/3773792033 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00e070 con 0x7f66b40cd440 2023-11-04T05:28:26.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b159a700 1 -- 172.21.15.3:0/3773792033 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac00b9c0 con 0x7f66b40cd440 2023-11-04T05:28:26.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b159a700 1 -- 172.21.15.3:0/3773792033 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00bd70 con 0x7f66b40cd440 2023-11-04T05:28:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b359e700 1 -- 172.21.15.3:0/3773792033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b40cd440 msgr2=0x7f66b40cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.982+0000 7f66b359e700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b40cd440 0x7f66b40cd860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66b406b7e0 tx=0x7f66ac00b330 comp rx=0 tx=0).stop 2023-11-04T05:28:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 -- 172.21.15.3:0/3773792033 shutdown_connections 2023-11-04T05:28:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b40cd440 0x7f66b40cd860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 --2- 172.21.15.3:0/3773792033 >> v2:172.21.15.188:3300/0 conn(0x7f66b415b9c0 0x7f66b41601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 --2- 172.21.15.3:0/3773792033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 -- 172.21.15.3:0/3773792033 >> 172.21.15.3:0/3773792033 conn(0x7f66b406dde0 msgr2=0x7f66b406e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 -- 172.21.15.3:0/3773792033 shutdown_connections 2023-11-04T05:28:26.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.983+0000 7f66b359e700 1 -- 172.21.15.3:0/3773792033 wait complete. 2023-11-04T05:28:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.984+0000 7f66b359e700 1 Processor -- start 2023-11-04T05:28:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.984+0000 7f66b359e700 1 -- start start 2023-11-04T05:28:26.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b40cd440 0x7f66b415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 0x7f66b415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b415bda0 con 0x7f66b415b9c0 2023-11-04T05:28:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b415cc00 con 0x7f66b40cd440 2023-11-04T05:28:26.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b415bff0 con 0x7f66b40b6260 2023-11-04T05:28:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 0x7f66b415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 0x7f66b415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51510/0 (socket says 172.21.15.3:51510) 2023-11-04T05:28:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 -- 172.21.15.3:0/1985886803 learned_addr learned my addr 172.21.15.3:0/1985886803 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b259c700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b1d9b700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b40cd440 0x7f66b415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:26.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 -- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 msgr2=0x7f66b40c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 -- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b40cd440 msgr2=0x7f66b415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b40cd440 0x7f66b415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 -- 172.21.15.3:0/1985886803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009b30 con 0x7f66b415b9c0 2023-11-04T05:28:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b1d9b700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b40cd440 0x7f66b415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b2d9d700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 0x7f66b415d090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00b370 tx=0x7f66ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:26.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac004310 con 0x7f66b415b9c0 2023-11-04T05:28:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac004470 con 0x7f66b415b9c0 2023-11-04T05:28:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b415c2d0 con 0x7f66b415b9c0 2023-11-04T05:28:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.985+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac004820 con 0x7f66b415b9c0 2023-11-04T05:28:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.986+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b415c890 con 0x7f66b415b9c0 2023-11-04T05:28:26.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.986+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b415cdb0 con 0x7f66b415b9c0 2023-11-04T05:28:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.987+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66ac01c440 con 0x7f66b415b9c0 2023-11-04T05:28:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.987+0000 7f66a2ffd700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669406af00 0x7f669406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.987+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66ac08b130 con 0x7f66b415b9c0 2023-11-04T05:28:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.987+0000 7f66b259c700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669406af00 0x7f669406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.988+0000 7f66b259c700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669406af00 0x7f669406d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66a800ab40 tx=0x7f66a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:26.992+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f66ac055f80 con 0x7f66b415b9c0 2023-11-04T05:28:27.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.240+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66b404f730 con 0x7f66b415b9c0 2023-11-04T05:28:27.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.241+0000 7f66a2ffd700 1 -- 172.21.15.3:0/1985886803 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f66ac0595a0 con 0x7f66b415b9c0 2023-11-04T05:28:27.243 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:27.244 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:26.265134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-04T05:28:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669406af00 msgr2=0x7f669406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669406af00 0x7f669406d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66a800ab40 tx=0x7f66a8008040 comp rx=0 tx=0).stop 2023-11-04T05:28:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 msgr2=0x7f66b415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 0x7f66b415d090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00b370 tx=0x7f66ac004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 shutdown_connections 2023-11-04T05:28:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669406af00 0x7f669406d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66b415b9c0 0x7f66b415d090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b6260 0x7f66b40c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 --2- 172.21.15.3:0/1985886803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b40cd440 0x7f66b415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 >> 172.21.15.3:0/1985886803 conn(0x7f66b406dde0 msgr2=0x7f66b4159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:27.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 shutdown_connections 2023-11-04T05:28:27.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.243+0000 7f66b359e700 1 -- 172.21.15.3:0/1985886803 wait complete. 2023-11-04T05:28:27.248 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-11-04T05:28:27.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:27.350 INFO:tasks.ceph.mds.d.smithi188.stderr:2023-11-04T05:28:27.349+0000 7fcaeebe5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-04T05:28:27.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.379+0000 7f73a08ba700 1 Processor -- start 2023-11-04T05:28:27.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.379+0000 7f73a08ba700 1 -- start start 2023-11-04T05:28:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.380+0000 7f73a08ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.380+0000 7f73a08ba700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f739c15b400 0x7f739c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.380+0000 7f73a08ba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c15af70 con 0x7f739c0cf710 2023-11-04T05:28:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.380+0000 7f73a08ba700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739c0cde10 con 0x7f739c15b400 2023-11-04T05:28:27.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.380+0000 7f73a08ba700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c072d30 con 0x7f739c0cf330 2023-11-04T05:28:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f7399d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f739c0cf710 0x7f739c0d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56862/0 (socket says 172.21.15.3:56862) 2023-11-04T05:28:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f7399d9b700 1 -- 172.21.15.3:0/3916726193 learned_addr learned my addr 172.21.15.3:0/3916726193 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739ad9d700 1 --2- 172.21.15.3:0/3916726193 >> v2:172.21.15.188:3300/0 conn(0x7f739c15b400 0x7f739c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739a59c700 1 --2- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739959a700 1 -- 172.21.15.3:0/3916726193 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629820669 0 0) 0x7f739c15af70 con 0x7f739c0cf710 2023-11-04T05:28:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739959a700 1 -- 172.21.15.3:0/3916726193 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73880035b0 con 0x7f739c0cf710 2023-11-04T05:28:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739959a700 1 -- 172.21.15.3:0/3916726193 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1183690200 0 0) 0x7f73880035b0 con 0x7f739c0cf710 2023-11-04T05:28:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739ad9d700 1 -- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 msgr2=0x7f739c0ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739ad9d700 1 --2- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739ad9d700 1 -- 172.21.15.3:0/3916726193 >> v1:172.21.15.3:6789/0 conn(0x7f739c0cf710 legacy=0x7f739c0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739ad9d700 1 -- 172.21.15.3:0/3916726193 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c0cd570 con 0x7f739c15b400 2023-11-04T05:28:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739ad9d700 1 --2- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c15b890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7394009d90 tx=0x7f739400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c4d1632a0cda89 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.381+0000 7f739959a700 1 -- 172.21.15.3:0/3916726193 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739400e070 con 0x7f739c15b400 2023-11-04T05:28:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f739959a700 1 -- 172.21.15.3:0/3916726193 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394003a20 con 0x7f739c15b400 2023-11-04T05:28:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f739959a700 1 -- 172.21.15.3:0/3916726193 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73840017f0 con 0x7f739c15b400 2023-11-04T05:28:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 -- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 msgr2=0x7f739c15b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 --2- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c15b890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7394009d90 tx=0x7f739400b330 comp rx=0 tx=0).stop 2023-11-04T05:28:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 -- 172.21.15.3:0/3916726193 shutdown_connections 2023-11-04T05:28:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 --2- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c15b890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 --2- 172.21.15.3:0/3916726193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 -- 172.21.15.3:0/3916726193 >> 172.21.15.3:0/3916726193 conn(0x7f739c06df90 msgr2=0x7f739c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 -- 172.21.15.3:0/3916726193 shutdown_connections 2023-11-04T05:28:27.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.382+0000 7f73a08ba700 1 -- 172.21.15.3:0/3916726193 wait complete. 2023-11-04T05:28:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 Processor -- start 2023-11-04T05:28:27.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 -- start start 2023-11-04T05:28:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 0x7f739c159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c1f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c157720 con 0x7f739c0cf710 2023-11-04T05:28:27.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c1f3340 con 0x7f739c15b400 2023-11-04T05:28:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f73a08ba700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c155470 con 0x7f739c0cf330 2023-11-04T05:28:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f7399d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 0x7f739c159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.384+0000 7f7399d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 0x7f739c159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51522/0 (socket says 172.21.15.3:51522) 2023-11-04T05:28:27.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 -- 172.21.15.3:0/4223345110 learned_addr learned my addr 172.21.15.3:0/4223345110 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f739a59c700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f739ad9d700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c1f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 -- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 msgr2=0x7f739c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 -- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 msgr2=0x7f739c1f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c1f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 -- 172.21.15.3:0/4223345110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009a70 con 0x7f739c0cf710 2023-11-04T05:28:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7399d9b700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 0x7f739c159ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f738400c750 tx=0x7f738400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73840074e0 con 0x7f739c0cf710 2023-11-04T05:28:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384007640 con 0x7f739c0cf710 2023-11-04T05:28:27.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c1f35e0 con 0x7f739c0cf710 2023-11-04T05:28:27.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384005a50 con 0x7f739c0cf710 2023-11-04T05:28:27.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.385+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c1f3ba0 con 0x7f739c0cf710 2023-11-04T05:28:27.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.386+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c1552d0 con 0x7f739c0cf710 2023-11-04T05:28:27.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.387+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7384005bb0 con 0x7f739c0cf710 2023-11-04T05:28:27.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.387+0000 7f7392ffd700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f737c06af00 0x7f737c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.387+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7384088f40 con 0x7f739c0cf710 2023-11-04T05:28:27.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.387+0000 7f739a59c700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f737c06af00 0x7f737c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.388+0000 7f739a59c700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f737c06af00 0x7f737c06d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f739c0c3f70 tx=0x7f738c009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.391+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f739c1552d0 con 0x7f739c0cf710 2023-11-04T05:28:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.593+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f739c155690 con 0x7f739c0cf710 2023-11-04T05:28:27.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.594+0000 7f7392ffd700 1 -- 172.21.15.3:0/4223345110 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f739c155690 con 0x7f739c0cf710 2023-11-04T05:28:27.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:27.597 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:27.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f737c06af00 msgr2=0x7f737c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f737c06af00 0x7f737c06d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f739c0c3f70 tx=0x7f738c009210 comp rx=0 tx=0).stop 2023-11-04T05:28:27.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 msgr2=0x7f739c159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 0x7f739c159ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f738400c750 tx=0x7f738400ca60 comp rx=0 tx=0).stop 2023-11-04T05:28:27.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 shutdown_connections 2023-11-04T05:28:27.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f737c06af00 0x7f737c06d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f739c15b400 0x7f739c1f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f739c0cf330 0x7f739c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 --2- 172.21.15.3:0/4223345110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f739c0cf710 0x7f739c159ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 >> 172.21.15.3:0/4223345110 conn(0x7f739c06df90 msgr2=0x7f739c0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:27.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 shutdown_connections 2023-11-04T05:28:27.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.596+0000 7f73a08ba700 1 -- 172.21.15.3:0/4223345110 wait complete. 2023-11-04T05:28:27.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:27.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.733+0000 7f0fb5f70700 1 Processor -- start 2023-11-04T05:28:27.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.733+0000 7f0fb5f70700 1 -- start start 2023-11-04T05:28:27.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0fb5f70700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb006ab20 0x7f0fb00728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0fb5f70700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fb006af00 0x7f0fb00d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0fb5f70700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb0072df0 con 0x7f0fb00cc7a0 2023-11-04T05:28:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0fb5f70700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00cf050 con 0x7f0fb006af00 2023-11-04T05:28:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0fb5f70700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00cda00 con 0x7f0fb006ab20 2023-11-04T05:28:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0faffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fb006af00 0x7f0fb00d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0faffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fb006af00 0x7f0fb00d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51532/0 (socket says 172.21.15.3:51532) 2023-11-04T05:28:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.734+0000 7f0faffff700 1 -- 172.21.15.3:0/1637320533 learned_addr learned my addr 172.21.15.3:0/1637320533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0faf7fe700 1 --2- 172.21.15.3:0/1637320533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb006ab20 0x7f0fb00728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0faffff700 1 -- 172.21.15.3:0/1637320533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb006ab20 msgr2=0x7f0fb00728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0faffff700 1 --2- 172.21.15.3:0/1637320533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb006ab20 0x7f0fb00728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0faffff700 1 -- 172.21.15.3:0/1637320533 >> v1:172.21.15.188:6789/0 conn(0x7f0fb00cc7a0 legacy=0x7f0fb00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0faffff700 1 -- 172.21.15.3:0/1637320533 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00cdce0 con 0x7f0fb006af00 2023-11-04T05:28:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0faffff700 1 --2- 172.21.15.3:0/1637320533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006af00 0x7f0fb00d0440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009e50 tx=0x7f0fa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592298ac7821c964 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0fae7fc700 1 -- 172.21.15.3:0/1637320533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800e070 con 0x7f0fb006af00 2023-11-04T05:28:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0fae7fc700 1 -- 172.21.15.3:0/1637320533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa800b910 con 0x7f0fb006af00 2023-11-04T05:28:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.735+0000 7f0fae7fc700 1 -- 172.21.15.3:0/1637320533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800bcc0 con 0x7f0fb006af00 2023-11-04T05:28:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 -- 172.21.15.3:0/1637320533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006af00 msgr2=0x7f0fb00d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/1637320533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006af00 0x7f0fb00d0440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009e50 tx=0x7f0fa800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 -- 172.21.15.3:0/1637320533 shutdown_connections 2023-11-04T05:28:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/1637320533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006af00 0x7f0fb00d0440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/1637320533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb006ab20 0x7f0fb00728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 -- 172.21.15.3:0/1637320533 >> 172.21.15.3:0/1637320533 conn(0x7f0fb006df90 msgr2=0x7f0fb006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 -- 172.21.15.3:0/1637320533 shutdown_connections 2023-11-04T05:28:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.736+0000 7f0fb5f70700 1 -- 172.21.15.3:0/1637320533 wait complete. 2023-11-04T05:28:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.737+0000 7f0fb5f70700 1 Processor -- start 2023-11-04T05:28:27.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.737+0000 7f0fb5f70700 1 -- start start 2023-11-04T05:28:27.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fb5f70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 0x7f0fb014fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fb5f70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb006af00 0x7f0fb00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fb5f70700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb00cc7a0 0x7f0fb00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fb5f70700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00ae0f0 con 0x7f0fb006ab20 2023-11-04T05:28:27.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fb5f70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0152a80 con 0x7f0fb006af00 2023-11-04T05:28:27.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fb5f70700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00ad790 con 0x7f0fb00cc7a0 2023-11-04T05:28:27.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 0x7f0fb014fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 0x7f0fb014fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51534/0 (socket says 172.21.15.3:51534) 2023-11-04T05:28:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 -- 172.21.15.3:0/2926586193 learned_addr learned my addr 172.21.15.3:0/2926586193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 -- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb00cc7a0 msgr2=0x7f0fb00c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faffff700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb00cc7a0 0x7f0fb00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faeffd700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb006af00 0x7f0fb00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb00cc7a0 0x7f0fb00c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 -- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb006af00 msgr2=0x7f0fb00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb006af00 0x7f0fb00c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 -- 172.21.15.3:0/2926586193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8009b30 con 0x7f0fb006ab20 2023-11-04T05:28:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0faf7fe700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 0x7f0fb014fe90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000e9e0 tx=0x7f0fa000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0009710 con 0x7f0fb006ab20 2023-11-04T05:28:27.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.738+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa000c850 con 0x7f0fb006ab20 2023-11-04T05:28:27.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.739+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb00ada30 con 0x7f0fb006ab20 2023-11-04T05:28:27.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.739+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa000cbc0 con 0x7f0fb006ab20 2023-11-04T05:28:27.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.739+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb01503d0 con 0x7f0fb006ab20 2023-11-04T05:28:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.739+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb0072090 con 0x7f0fb006ab20 2023-11-04T05:28:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.740+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0fa000cd20 con 0x7f0fb006ab20 2023-11-04T05:28:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.740+0000 7f0fa7fff700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0f9006af00 0x7f0f9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.740+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0fa0089910 con 0x7f0fb006ab20 2023-11-04T05:28:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.741+0000 7f0faeffd700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0f9006af00 0x7f0f9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.741+0000 7f0faeffd700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0f9006af00 0x7f0f9006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00c3a00 tx=0x7f0f98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.745+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f0fb0072090 con 0x7f0fb006ab20 2023-11-04T05:28:27.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.949+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fb00ae360 con 0x7f0fb006ab20 2023-11-04T05:28:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.950+0000 7f0fa7fff700 1 -- 172.21.15.3:0/2926586193 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f0fa00581f0 con 0x7f0fb006ab20 2023-11-04T05:28:27.952 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:27.953 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:27.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0f9006af00 msgr2=0x7f0f9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0f9006af00 0x7f0f9006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00c3a00 tx=0x7f0f98009040 comp rx=0 tx=0).stop 2023-11-04T05:28:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 msgr2=0x7f0fb014fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 0x7f0fb014fe90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000e9e0 tx=0x7f0fa000eda0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 shutdown_connections 2023-11-04T05:28:27.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0f9006af00 0x7f0f9006d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb00cc7a0 0x7f0fb00c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb006ab20 0x7f0fb014fe90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 --2- 172.21.15.3:0/2926586193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb006af00 0x7f0fb00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:27.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 >> 172.21.15.3:0/2926586193 conn(0x7f0fb006df90 msgr2=0x7f0fb00c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:27.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 shutdown_connections 2023-11-04T05:28:27.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:27.952+0000 7f0fb5f70700 1 -- 172.21.15.3:0/2926586193 wait complete. 2023-11-04T05:28:27.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.104+0000 7f92fb015700 1 Processor -- start 2023-11-04T05:28:28.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.104+0000 7f92fb015700 1 -- start start 2023-11-04T05:28:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.104+0000 7f92fb015700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92f40cf330 0x7f92f40ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.104+0000 7f92fb015700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 0x7f92f415b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.104+0000 7f92fb015700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f415af70 con 0x7f92f40cf710 2023-11-04T05:28:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92fb015700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f40cde10 con 0x7f92f40cf330 2023-11-04T05:28:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92fb015700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4072d30 con 0x7f92f415b400 2023-11-04T05:28:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92f40cf710 0x7f92f40d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56890/0 (socket says 172.21.15.3:56890) 2023-11-04T05:28:28.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f3fff700 1 -- 172.21.15.3:0/2278776871 learned_addr learned my addr 172.21.15.3:0/2278776871 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f95b2700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 0x7f92f415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f8db1700 1 --2- 172.21.15.3:0/2278776871 >> v2:172.21.15.188:3300/0 conn(0x7f92f40cf330 0x7f92f40ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615243163 0 0) 0x7f92f415af70 con 0x7f92f40cf710 2023-11-04T05:28:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e40035b0 con 0x7f92f40cf710 2023-11-04T05:28:28.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1457832158 0 0) 0x7f92e40035b0 con 0x7f92f40cf710 2023-11-04T05:28:28.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 msgr2=0x7f92f415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:28.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 0x7f92f415b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 >> v2:172.21.15.188:3300/0 conn(0x7f92f40cf330 msgr2=0x7f92f40ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:28.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 --2- 172.21.15.3:0/2278776871 >> v2:172.21.15.188:3300/0 conn(0x7f92f40cf330 0x7f92f40ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40cd570 con 0x7f92f40cf710 2023-11-04T05:28:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f92e00017f0 con 0x7f92f40cf710 2023-11-04T05:28:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 >> v1:172.21.15.3:6789/0 conn(0x7f92f40cf710 legacy=0x7f92f40d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 0x7f92e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e4004d90 0x7f92e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40cde10 con 0x7f92e4003a00 2023-11-04T05:28:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4072d30 con 0x7f92e4004d90 2023-11-04T05:28:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e400ba10 con 0x7f92f415b400 2023-11-04T05:28:28.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f8db1700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.105+0000 7f92f3fff700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 0x7f92e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f95b2700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e4004d90 0x7f92e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f8db1700 1 -- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 msgr2=0x7f92e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f8db1700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f415b400 0x7f92e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f8db1700 1 -- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e4004d90 msgr2=0x7f92e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f8db1700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e4004d90 0x7f92e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f8db1700 1 -- 172.21.15.3:0/2278776871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40035b0 con 0x7f92e4003a00 2023-11-04T05:28:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f8db1700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92e4003e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f92e800de70 tx=0x7f92e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc827a2142511a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e800d2d0 con 0x7f92e4003a00 2023-11-04T05:28:28.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e8013040 con 0x7f92e4003a00 2023-11-04T05:28:28.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f37fe700 1 -- 172.21.15.3:0/2278776871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8012b30 con 0x7f92e4003a00 2023-11-04T05:28:28.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92fb015700 1 -- 172.21.15.3:0/2278776871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 msgr2=0x7f92e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92fb015700 1 --2- 172.21.15.3:0/2278776871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92e4003e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f92e800de70 tx=0x7f92e4005880 comp rx=0 tx=0).stop 2023-11-04T05:28:28.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.106+0000 7f92f95b2700 1 -- 172.21.15.3:0/2278776871 reap_dead start 2023-11-04T05:28:28.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.107+0000 7f92fb015700 1 -- 172.21.15.3:0/2278776871 shutdown_connections 2023-11-04T05:28:28.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.107+0000 7f92fb015700 1 -- 172.21.15.3:0/2278776871 >> 172.21.15.3:0/2278776871 conn(0x7f92f406df90 msgr2=0x7f92f406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:28.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.107+0000 7f92fb015700 1 -- 172.21.15.3:0/2278776871 shutdown_connections 2023-11-04T05:28:28.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.107+0000 7f92fb015700 1 -- 172.21.15.3:0/2278776871 wait complete. 2023-11-04T05:28:28.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.108+0000 7f92fb015700 1 Processor -- start 2023-11-04T05:28:28.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.108+0000 7f92fb015700 1 -- start start 2023-11-04T05:28:28.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92fb015700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92f4159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92fb015700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f40b3a80 0x7f92f415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92fb015700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f415d530 0x7f92f41f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92fb015700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f415d9c0 con 0x7f92e4003a00 2023-11-04T05:28:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92fb015700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40ca7a0 con 0x7f92f415d530 2023-11-04T05:28:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92fb015700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4159e80 con 0x7f92f40b3a80 2023-11-04T05:28:28.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92f8db1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92f4159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92f8db1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92f4159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51554/0 (socket says 172.21.15.3:51554) 2023-11-04T05:28:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.109+0000 7f92f8db1700 1 -- 172.21.15.3:0/536674536 learned_addr learned my addr 172.21.15.3:0/536674536 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f3fff700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f40b3a80 0x7f92f415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f95b2700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f415d530 0x7f92f41f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f8db1700 1 -- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f40b3a80 msgr2=0x7f92f415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f8db1700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f40b3a80 0x7f92f415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f8db1700 1 -- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f415d530 msgr2=0x7f92f41f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f8db1700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f415d530 0x7f92f41f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f8db1700 1 -- 172.21.15.3:0/536674536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f41eea20 con 0x7f92e4003a00 2023-11-04T05:28:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f95b2700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f415d530 0x7f92f41f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:28.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f8db1700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92f4159940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92e800efd0 tx=0x7f92e800d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:28.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8012e20 con 0x7f92e4003a00 2023-11-04T05:28:28.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e8026070 con 0x7f92e4003a00 2023-11-04T05:28:28.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f40c8ab0 con 0x7f92e4003a00 2023-11-04T05:28:28.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e802d6c0 con 0x7f92e4003a00 2023-11-04T05:28:28.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.110+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f40c9070 con 0x7f92e4003a00 2023-11-04T05:28:28.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.111+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f4074df0 con 0x7f92e4003a00 2023-11-04T05:28:28.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.112+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92e8018480 con 0x7f92e4003a00 2023-11-04T05:28:28.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.112+0000 7f92f17fa700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f92d806af00 0x7f92d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.112+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f92e8028070 con 0x7f92e4003a00 2023-11-04T05:28:28.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.112+0000 7f92f3fff700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f92d806af00 0x7f92d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.113+0000 7f92f3fff700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f92d806af00 0x7f92d806d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000ef0 tx=0x7f92e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:28.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.117+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f92e805a2a0 con 0x7f92e4003a00 2023-11-04T05:28:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.361+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92f406e510 con 0x7f92e4003a00 2023-11-04T05:28:28.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.362+0000 7f92f17fa700 1 -- 172.21.15.3:0/536674536 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f92e805d8c0 con 0x7f92e4003a00 2023-11-04T05:28:28.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:26.265134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-04T05:28:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f92d806af00 msgr2=0x7f92d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f92d806af00 0x7f92d806d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000ef0 tx=0x7f92e0009040 comp rx=0 tx=0).stop 2023-11-04T05:28:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 msgr2=0x7f92f4159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92f4159940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92e800efd0 tx=0x7f92e800d620 comp rx=0 tx=0).stop 2023-11-04T05:28:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 shutdown_connections 2023-11-04T05:28:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f92d806af00 0x7f92d806d3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f415d530 0x7f92f41f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92e4003a00 0x7f92f4159940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 --2- 172.21.15.3:0/536674536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f40b3a80 0x7f92f415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 >> 172.21.15.3:0/536674536 conn(0x7f92f406df90 msgr2=0x7f92f415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 shutdown_connections 2023-11-04T05:28:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.364+0000 7f92fb015700 1 -- 172.21.15.3:0/536674536 wait complete. 2023-11-04T05:28:28.426 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-11-04T05:28:28.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:28.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.551+0000 7f8980d8b700 1 Processor -- start 2023-11-04T05:28:28.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.551+0000 7f8980d8b700 1 -- start start 2023-11-04T05:28:28.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f8980d8b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f897c06ab20 0x7f897c0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f8980d8b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c06af00 0x7f897c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f8980d8b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c072df0 con 0x7f897c0cc7a0 2023-11-04T05:28:28.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f8980d8b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c0cf050 con 0x7f897c06ab20 2023-11-04T05:28:28.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f8980d8b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0cda00 con 0x7f897c06af00 2023-11-04T05:28:28.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f897c06ab20 0x7f897c0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f897c06ab20 0x7f897c0728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51562/0 (socket says 172.21.15.3:51562) 2023-11-04T05:28:28.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 -- 172.21.15.3:0/2169744953 learned_addr learned my addr 172.21.15.3:0/2169744953 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:28.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897ad9d700 1 --2- 172.21.15.3:0/2169744953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c06af00 0x7f897c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 -- 172.21.15.3:0/2169744953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c06af00 msgr2=0x7f897c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:28.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 --2- 172.21.15.3:0/2169744953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c06af00 0x7f897c0d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 -- 172.21.15.3:0/2169744953 >> v1:172.21.15.188:6789/0 conn(0x7f897c0cc7a0 legacy=0x7f897c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 -- 172.21.15.3:0/2169744953 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0cdce0 con 0x7f897c06ab20 2023-11-04T05:28:28.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.552+0000 7f897a59c700 1 --2- 172.21.15.3:0/2169744953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0728b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098f0 tx=0x7f896c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489bfc44cf2f4a60 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f897959a700 1 -- 172.21.15.3:0/2169744953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c011830 con 0x7f897c06ab20 2023-11-04T05:28:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f897959a700 1 -- 172.21.15.3:0/2169744953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c011990 con 0x7f897c06ab20 2023-11-04T05:28:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f897959a700 1 -- 172.21.15.3:0/2169744953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c011d20 con 0x7f897c06ab20 2023-11-04T05:28:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 -- 172.21.15.3:0/2169744953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 msgr2=0x7f897c0728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 --2- 172.21.15.3:0/2169744953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0728b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098f0 tx=0x7f896c009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 -- 172.21.15.3:0/2169744953 shutdown_connections 2023-11-04T05:28:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 --2- 172.21.15.3:0/2169744953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c06af00 0x7f897c0d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 --2- 172.21.15.3:0/2169744953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0728b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 -- 172.21.15.3:0/2169744953 >> 172.21.15.3:0/2169744953 conn(0x7f897c06df90 msgr2=0x7f897c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 -- 172.21.15.3:0/2169744953 shutdown_connections 2023-11-04T05:28:28.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.553+0000 7f8980d8b700 1 -- 172.21.15.3:0/2169744953 wait complete. 2023-11-04T05:28:28.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 Processor -- start 2023-11-04T05:28:28.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 -- start start 2023-11-04T05:28:28.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f897c06af00 0x7f897c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c0cc7a0 0x7f897c14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c5100 con 0x7f897c06ab20 2023-11-04T05:28:28.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c5bd0 con 0x7f897c06af00 2023-11-04T05:28:28.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f8980d8b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c4590 con 0x7f897c0cc7a0 2023-11-04T05:28:28.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.555+0000 7f897a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51568/0 (socket says 172.21.15.3:51568) 2023-11-04T05:28:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 -- 172.21.15.3:0/3825530780 learned_addr learned my addr 172.21.15.3:0/3825530780 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897ad9d700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c0cc7a0 0x7f897c14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f8979d9b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f897c06af00 0x7f897c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 -- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c0cc7a0 msgr2=0x7f897c14c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c0cc7a0 0x7f897c14c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 -- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f897c06af00 msgr2=0x7f897c15c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f897c06af00 0x7f897c15c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 -- 172.21.15.3:0/3825530780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c009550 con 0x7f897c06ab20 2023-11-04T05:28:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f897a59c700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0c4050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f896c009c40 tx=0x7f896c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c004380 con 0x7f897c06ab20 2023-11-04T05:28:28.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c01f070 con 0x7f897c06ab20 2023-11-04T05:28:28.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0036a0 con 0x7f897c06ab20 2023-11-04T05:28:28.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c0c4860 con 0x7f897c06ab20 2023-11-04T05:28:28.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.556+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c0c4e20 con 0x7f897c06ab20 2023-11-04T05:28:28.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.557+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c0c59a0 con 0x7f897c06ab20 2023-11-04T05:28:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.558+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f896c021030 con 0x7f897c06ab20 2023-11-04T05:28:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.558+0000 7f896affd700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f895c06af00 0x7f895c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.558+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f896c08abb0 con 0x7f897c06ab20 2023-11-04T05:28:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.558+0000 7f8979d9b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f895c06af00 0x7f895c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.559+0000 7f8979d9b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f895c06af00 0x7f895c06d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f897c0ad2a0 tx=0x7f8970009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.563+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f897c0c59a0 con 0x7f897c06ab20 2023-11-04T05:28:28.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.765+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f897c0c59a0 con 0x7f897c06ab20 2023-11-04T05:28:28.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.767+0000 7f896affd700 1 -- 172.21.15.3:0/3825530780 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f897c0c59a0 con 0x7f897c06ab20 2023-11-04T05:28:28.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:28.770 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:28.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f895c06af00 msgr2=0x7f895c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f895c06af00 0x7f895c06d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f897c0ad2a0 tx=0x7f8970009040 comp rx=0 tx=0).stop 2023-11-04T05:28:28.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 msgr2=0x7f897c0c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:28.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0c4050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f896c009c40 tx=0x7f896c004750 comp rx=0 tx=0).stop 2023-11-04T05:28:28.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 shutdown_connections 2023-11-04T05:28:28.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f895c06af00 0x7f895c06d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f897c0cc7a0 0x7f897c14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f897c06ab20 0x7f897c0c4050 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 --2- 172.21.15.3:0/3825530780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f897c06af00 0x7f897c15c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:28.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 >> 172.21.15.3:0/3825530780 conn(0x7f897c06df90 msgr2=0x7f897c1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:28.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 shutdown_connections 2023-11-04T05:28:28.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:28.769+0000 7f8980d8b700 1 -- 172.21.15.3:0/3825530780 wait complete. 2023-11-04T05:28:29.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-04T05:28:20.853748+0000', 'modified': '2023-11-04T05:28:26.265134+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6837/3609354532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6835', 'nonce': 3609354532}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3609354532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-04T05:28:29.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-04T05:28:29.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-04T05:28:29.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.313+0000 7f1cdf4df700 1 Processor -- start 2023-11-04T05:28:29.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.313+0000 7f1cdf4df700 1 -- start start 2023-11-04T05:28:29.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdf4df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cd815de50 0x7f1cd806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdf4df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd80ce3a0 con 0x7f1cd806c240 2023-11-04T05:28:29.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdf4df700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd806e030 con 0x7f1cd806db90 2023-11-04T05:28:29.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdf4df700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80cdd50 con 0x7f1cd815de50 2023-11-04T05:28:29.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdd27b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cd815de50 0x7f1cd806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdda7c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1cd806c240 0x7f1cd80ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56928/0 (socket says 172.21.15.3:56928) 2023-11-04T05:28:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.314+0000 7f1cdda7c700 1 -- 172.21.15.3:0/486473815 learned_addr learned my addr 172.21.15.3:0/486473815 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1cdd27b700 1 -- 172.21.15.3:0/486473815 >> v1:172.21.15.188:6789/0 conn(0x7f1cd806db90 legacy=0x7f1cd806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1cdd27b700 1 -- 172.21.15.3:0/486473815 >> v1:172.21.15.3:6789/0 conn(0x7f1cd806c240 legacy=0x7f1cd80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1cdd27b700 1 -- 172.21.15.3:0/486473815 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd80ce030 con 0x7f1cd815de50 2023-11-04T05:28:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1ccffff700 1 -- 172.21.15.3:0/486473815 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144181489 0 0) 0x7f1cd80ce3a0 con 0x7f1cd806c240 2023-11-04T05:28:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/486473815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd815de50 0x7f1cd806d650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009840 tx=0x7f1cc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf380db65570307 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1ccffff700 1 -- 172.21.15.3:0/486473815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8004230 con 0x7f1cd815de50 2023-11-04T05:28:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1ccffff700 1 -- 172.21.15.3:0/486473815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc8004390 con 0x7f1cd815de50 2023-11-04T05:28:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1ccffff700 1 -- 172.21.15.3:0/486473815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80036a0 con 0x7f1cd815de50 2023-11-04T05:28:29.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1cdf4df700 1 -- 172.21.15.3:0/486473815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd815de50 msgr2=0x7f1cd806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.315+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/486473815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd815de50 0x7f1cd806d650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009840 tx=0x7f1cc8009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:29.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.316+0000 7f1cdf4df700 1 -- 172.21.15.3:0/486473815 shutdown_connections 2023-11-04T05:28:29.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.316+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/486473815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd815de50 0x7f1cd806d650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.316+0000 7f1cdf4df700 1 -- 172.21.15.3:0/486473815 >> 172.21.15.3:0/486473815 conn(0x7f1cd80716c0 msgr2=0x7f1cd8071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.316+0000 7f1cdf4df700 1 -- 172.21.15.3:0/486473815 shutdown_connections 2023-11-04T05:28:29.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.316+0000 7f1cdf4df700 1 -- 172.21.15.3:0/486473815 wait complete. 2023-11-04T05:28:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.317+0000 7f1cdf4df700 1 Processor -- start 2023-11-04T05:28:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.317+0000 7f1cdf4df700 1 -- start start 2023-11-04T05:28:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.317+0000 7f1cdf4df700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd806c240 0x7f1cd8153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.317+0000 7f1cdf4df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 0x7f1cd80c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdf4df700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd815de50 0x7f1cd80c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdf4df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80b2f60 con 0x7f1cd806db90 2023-11-04T05:28:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdf4df700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80c8750 con 0x7f1cd806c240 2023-11-04T05:28:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdf4df700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80c7bd0 con 0x7f1cd815de50 2023-11-04T05:28:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 0x7f1cd80c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 0x7f1cd80c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51584/0 (socket says 172.21.15.3:51584) 2023-11-04T05:28:29.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 -- 172.21.15.3:0/1213187825 learned_addr learned my addr 172.21.15.3:0/1213187825 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd806c240 0x7f1cd8153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdda7c700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd815de50 0x7f1cd80c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 -- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd815de50 msgr2=0x7f1cd80c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd815de50 0x7f1cd80c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 -- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd806c240 msgr2=0x7f1cd8153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd806c240 0x7f1cd8153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 -- 172.21.15.3:0/1213187825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80094a0 con 0x7f1cd806db90 2023-11-04T05:28:29.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd806c240 0x7f1cd8153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 0x7f1cd80c8d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c4d0 tx=0x7f1cd400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1ccdffb700 1 -- 172.21.15.3:0/1213187825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4011790 con 0x7f1cd806db90 2023-11-04T05:28:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1ccdffb700 1 -- 172.21.15.3:0/1213187825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40118f0 con 0x7f1cd806db90 2023-11-04T05:28:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd80c7eb0 con 0x7f1cd806db90 2023-11-04T05:28:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1ccdffb700 1 -- 172.21.15.3:0/1213187825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4011ca0 con 0x7f1cd806db90 2023-11-04T05:28:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.318+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd80c8470 con 0x7f1cd806db90 2023-11-04T05:28:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.320+0000 7f1ccdffb700 1 -- 172.21.15.3:0/1213187825 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cd401c460 con 0x7f1cd806db90 2023-11-04T05:28:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.320+0000 7f1ccdffb700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1cbc06af00 0x7f1cbc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.320+0000 7f1ccdffb700 1 -- 172.21.15.3:0/1213187825 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1cd4089730 con 0x7f1cd806db90 2023-11-04T05:28:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.320+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1cbc06af00 0x7f1cbc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.321+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1cbc06af00 0x7f1cbc06d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80090b0 tx=0x7f1cc8013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.322+0000 7f1ceb6d9b80 1 Processor -- start 2023-11-04T05:28:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.322+0000 7f1ceb6d9b80 1 -- start start 2023-11-04T05:28:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073ec0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 0x565073ec0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073e90ad0 0x565073eaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073dfc6e0 con 0x565073e92c30 2023-11-04T05:28:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073df03b0 con 0x565073e906f0 2023-11-04T05:28:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x565073dedcd0 con 0x565073e90ad0 2023-11-04T05:28:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 0x565073ec0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 0x565073ec0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51594/0 (socket says 172.21.15.3:51594) 2023-11-04T05:28:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 -- 172.21.15.3:0/658210804 learned_addr learned my addr 172.21.15.3:0/658210804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073ec0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdda7c700 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073e90ad0 0x565073eaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 -- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073e90ad0 msgr2=0x565073eaaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073e90ad0 0x565073eaaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 -- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 msgr2=0x565073ec0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073ec0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 -- 172.21.15.3:0/658210804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565073c665e0 con 0x565073e92c30 2023-11-04T05:28:29.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.323+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 0x565073ec0990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400a040 tx=0x7f1cd405e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9723ad86f3bde server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1cc2ffd700 1 -- 172.21.15.3:0/658210804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd406f070 con 0x565073e92c30 2023-11-04T05:28:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1cc2ffd700 1 -- 172.21.15.3:0/658210804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40581b0 con 0x565073e92c30 2023-11-04T05:28:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1cc2ffd700 1 -- 172.21.15.3:0/658210804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd405c300 con 0x565073e92c30 2023-11-04T05:28:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/658210804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 msgr2=0x565073ec0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 0x565073ec0990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400a040 tx=0x7f1cd405e040 comp rx=0 tx=0).stop 2023-11-04T05:28:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/658210804 shutdown_connections 2023-11-04T05:28:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073ec0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073e90ad0 0x565073eaaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/658210804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e92c30 0x565073ec0990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/658210804 >> 172.21.15.3:0/658210804 conn(0x565073e0b660 msgr2=0x565073e963c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/658210804 shutdown_connections 2023-11-04T05:28:29.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.324+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/658210804 wait complete. 2023-11-04T05:28:29.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.325+0000 7f1ceb6d9b80 1 -- start start 2023-11-04T05:28:29.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.325+0000 7f1ceb6d9b80 1 Processor -- start 2023-11-04T05:28:29.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073edb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073ec3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e91020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073de95c0 con 0x565073e90ad0 2023-11-04T05:28:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073d850d0 con 0x565073e906f0 2023-11-04T05:28:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x565073d84d70 con 0x565073ea0620 2023-11-04T05:28:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1ceb6d9b80 10 client.0 _ll_register_callbacks cb 0x565073be17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-04T05:28:29.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073ec3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073ec3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51598/0 (socket says 172.21.15.3:51598) 2023-11-04T05:28:29.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 -- 172.21.15.3:0/2255281651 learned_addr learned my addr 172.21.15.3:0/2255281651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073edb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdda7c700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e91020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 -- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 msgr2=0x565073e91020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e91020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 -- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 msgr2=0x565073edb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073edb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 -- 172.21.15.3:0/2255281651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd80c8470 con 0x565073e90ad0 2023-11-04T05:28:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073ec3d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40879c0 tx=0x7f1cd401e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a46dab0da652d5d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-11-04T05:28:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.326+0000 7f1cb7fff700 1 -- 172.21.15.3:0/2255281651 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd406f040 con 0x565073e90ad0 2023-11-04T05:28:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1cb7fff700 1 -- 172.21.15.3:0/2255281651 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1cd40604b0 con 0x565073e90ad0 2023-11-04T05:28:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565073e88880 con 0x565073e90ad0 2023-11-04T05:28:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1cb7fff700 1 -- 172.21.15.3:0/2255281651 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cd405c300 con 0x565073e90ad0 2023-11-04T05:28:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 10 client.4437 fetch_fsmap learned FSMap version 7 2023-11-04T05:28:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x565073c12d90 con 0x565073e90ad0 2023-11-04T05:28:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1cb7fff700 1 -- 172.21.15.3:0/2255281651 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565073e88880 con 0x565073e90ad0 2023-11-04T05:28:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1cb7fff700 1 -- 172.21.15.3:0/2255281651 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1cd40576a0 con 0x565073e90ad0 2023-11-04T05:28:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 10 client.4437 fetch_fsmap finished waiting for FSMap version 7 2023-11-04T05:28:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 10 client.4437 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-04T05:28:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 20 client.4437 populate_metadata read hostname 'smithi003' 2023-11-04T05:28:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073ed3d60 0x565073e9e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 4 client.4437 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-04T05:28:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 --> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565073e64a30 con 0x565073ed3d60 2023-11-04T05:28:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.327+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073ed3d60 0x565073e9e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1cb7fff700 10 client.4437 ms_handle_connect on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073ed3d60 0x565073e9e470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=573907f565904d88 server_cookie=9a0dac99874a646 in_seq=0 out_seq=0 2023-11-04T05:28:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1cb7fff700 1 -- 172.21.15.3:0/2255281651 <== mds.0 v2:172.21.15.188:6835/3609354532 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565073e64a30 con 0x565073ed3d60 2023-11-04T05:28:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1cb7fff700 10 client.4437 handle_command_reply: tid=0 2023-11-04T05:28:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1ceb6d9b80 1 client.4437 shutdown 2023-11-04T05:28:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 msgr2=0x565073ec3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073ec3d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40879c0 tx=0x7f1cd401e070 comp rx=0 tx=0).stop 2023-11-04T05:28:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 shutdown_connections 2023-11-04T05:28:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.328+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073ed3d60 0x565073e9e470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073edb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e91020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1cb7fff700 0 client.4437 ms_handle_reset on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/2255281651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073ec3d00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 >> 172.21.15.3:0/2255281651 conn(0x565073e0b660 msgr2=0x565073edc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 shutdown_connections 2023-11-04T05:28:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/2255281651 wait complete. 2023-11-04T05:28:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.329+0000 7f1ceb6d9b80 20 client.4437 trim_cache size 0 max 16384 2023-11-04T05:28:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.347+0000 7f1ceb6d9b80 1 Processor -- start 2023-11-04T05:28:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.347+0000 7f1ceb6d9b80 1 -- start start 2023-11-04T05:28:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 0x565073e9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e90ad0 0x565073e934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073c688d0 con 0x565073e906f0 2023-11-04T05:28:29.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073ec3de0 con 0x565073e90ad0 2023-11-04T05:28:29.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x565073edb150 con 0x565073ea0620 2023-11-04T05:28:29.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 0x565073e9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 0x565073e9f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51604/0 (socket says 172.21.15.3:51604) 2023-11-04T05:28:29.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 -- 172.21.15.3:0/3102254756 learned_addr learned my addr 172.21.15.3:0/3102254756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 -- 172.21.15.3:0/3102254756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 msgr2=0x565073e93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 -- 172.21.15.3:0/3102254756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e90ad0 msgr2=0x565073e934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e90ad0 0x565073e934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.348+0000 7f1cdd27b700 1 -- 172.21.15.3:0/3102254756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565073ed4410 con 0x565073e906f0 2023-11-04T05:28:29.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 0x565073e9f980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfdb7a28c50f37d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1cc37fe700 1 -- 172.21.15.3:0/3102254756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8030040 con 0x565073e906f0 2023-11-04T05:28:29.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1cc37fe700 1 -- 172.21.15.3:0/3102254756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc8004500 con 0x565073e906f0 2023-11-04T05:28:29.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1cc37fe700 1 -- 172.21.15.3:0/3102254756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8003c80 con 0x565073e906f0 2023-11-04T05:28:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/3102254756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 msgr2=0x565073e9f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 0x565073e9f980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc8008030 comp rx=0 tx=0).stop 2023-11-04T05:28:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/3102254756 shutdown_connections 2023-11-04T05:28:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e906f0 0x565073e9f980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/3102254756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e90ad0 0x565073e934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/3102254756 >> 172.21.15.3:0/3102254756 conn(0x565073e0b660 msgr2=0x565073ec1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/3102254756 shutdown_connections 2023-11-04T05:28:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.349+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/3102254756 wait complete. 2023-11-04T05:28:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.350+0000 7f1ceb6d9b80 1 -- start start 2023-11-04T05:28:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.350+0000 7f1ceb6d9b80 1 Processor -- start 2023-11-04T05:28:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073f44810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073efd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073edbc10 con 0x565073e90ad0 2023-11-04T05:28:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x565073ed1cc0 con 0x565073e906f0 2023-11-04T05:28:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x565073e8f620 con 0x565073ea0620 2023-11-04T05:28:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1ceb6d9b80 10 client.0 _ll_register_callbacks cb 0x7f1cd4057b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-04T05:28:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073efd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073efd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51612/0 (socket says 172.21.15.3:51612) 2023-11-04T05:28:29.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 -- 172.21.15.3:0/80205561 learned_addr learned my addr 172.21.15.3:0/80205561 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073f44810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdda7c700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 -- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 msgr2=0x565073e934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 -- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 msgr2=0x565073f44810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073f44810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 -- 172.21.15.3:0/80205561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd80ce030 con 0x565073e90ad0 2023-11-04T05:28:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cdca7a700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073efd540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40546b0 tx=0x7f1cd401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658d9f4c5375a061 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.351+0000 7f1cb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-11-04T05:28:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1cb7fff700 1 -- 172.21.15.3:0/80205561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd406f040 con 0x565073e90ad0 2023-11-04T05:28:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1cb7fff700 1 -- 172.21.15.3:0/80205561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1cd40604b0 con 0x565073e90ad0 2023-11-04T05:28:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cd40581b0 con 0x565073e90ad0 2023-11-04T05:28:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1cb7fff700 1 -- 172.21.15.3:0/80205561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cd40039b0 con 0x565073e90ad0 2023-11-04T05:28:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 10 client.4443 fetch_fsmap learned FSMap version 7 2023-11-04T05:28:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x565073bfbc10 con 0x565073e90ad0 2023-11-04T05:28:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1cb7fff700 1 -- 172.21.15.3:0/80205561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40581b0 con 0x565073e90ad0 2023-11-04T05:28:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1cb7fff700 1 -- 172.21.15.3:0/80205561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1cd405e0b0 con 0x565073e90ad0 2023-11-04T05:28:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2023-11-04T05:28:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 10 client.4443 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-04T05:28:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 20 client.4443 populate_metadata read hostname 'smithi003' 2023-11-04T05:28:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073e93a00 0x565073ed0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 4 client.4443 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-04T05:28:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 --> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] -- command(tid 0: {"prefix": "status"}) v1 -- 0x565073de25a0 con 0x565073e93a00 2023-11-04T05:28:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.352+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073e93a00 0x565073ed0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1cb7fff700 10 client.4443 ms_handle_connect on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1cdd27b700 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073e93a00 0x565073ed0de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de1d4386ae070e59 server_cookie=2a05a24987ad3426 in_seq=0 out_seq=0 2023-11-04T05:28:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1cb7fff700 1 -- 172.21.15.3:0/80205561 <== mds.0 v2:172.21.15.188:6835/3609354532 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x565073de25a0 con 0x565073e93a00 2023-11-04T05:28:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1cb7fff700 10 client.4443 handle_command_reply: tid=0 2023-11-04T05:28:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 client.4443 shutdown 2023-11-04T05:28:29.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 msgr2=0x565073efd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073efd540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40546b0 tx=0x7f1cd401c000 comp rx=0 tx=0).stop 2023-11-04T05:28:29.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 shutdown_connections 2023-11-04T05:28:29.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x565073e93a00 0x565073ed0de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565073e906f0 0x565073f44810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1cb7fff700 0 client.4443 ms_handle_reset on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x565073ea0620 0x565073e934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 --2- 172.21.15.3:0/80205561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565073e90ad0 0x565073efd540 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 >> 172.21.15.3:0/80205561 conn(0x565073e0b660 msgr2=0x565073ec1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 shutdown_connections 2023-11-04T05:28:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 1 -- 172.21.15.3:0/80205561 wait complete. 2023-11-04T05:28:29.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.353+0000 7f1ceb6d9b80 20 client.4443 trim_cache size 0 max 16384 2023-11-04T05:28:29.375 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-11-04T05:28:29.375 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "7a64e299-05b0-4fd0-819f-2262a06846bb", 2023-11-04T05:28:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-11-04T05:28:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4288, 2023-11-04T05:28:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-11-04T05:28:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-11-04T05:28:29.376 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-11-04T05:28:29.377 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 7.9201715940000001, 2023-11-04T05:28:29.377 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 7, 2023-11-04T05:28:29.377 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 26, 2023-11-04T05:28:29.377 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 25, 2023-11-04T05:28:29.377 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 13.002284761 2023-11-04T05:28:29.378 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-11-04T05:28:29.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1cbc06af00 msgr2=0x7f1cbc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1cbc06af00 0x7f1cbc06d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80090b0 tx=0x7f1cc8013040 comp rx=0 tx=0).stop 2023-11-04T05:28:29.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 msgr2=0x7f1cd80c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 0x7f1cd80c8d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c4d0 tx=0x7f1cd400c890 comp rx=0 tx=0).stop 2023-11-04T05:28:29.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 shutdown_connections 2023-11-04T05:28:29.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1cbc06af00 0x7f1cbc06d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd806c240 0x7f1cd8153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd815de50 0x7f1cd80c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 --2- 172.21.15.3:0/1213187825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cd806db90 0x7f1cd80c8d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 >> 172.21.15.3:0/1213187825 conn(0x7f1cd80716c0 msgr2=0x7f1cd80ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 shutdown_connections 2023-11-04T05:28:29.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.354+0000 7f1cdf4df700 1 -- 172.21.15.3:0/1213187825 wait complete. 2023-11-04T05:28:29.382 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-04T05:28:29.385 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:29.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-04T05:28:29.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.512+0000 7fabee7fb700 1 Processor -- start 2023-11-04T05:28:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.512+0000 7fabee7fb700 1 -- start start 2023-11-04T05:28:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabee7fb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabe806ab20 0x7fabe80728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabee7fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabee7fb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe8072df0 con 0x7fabe806af00 2023-11-04T05:28:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabee7fb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe80cf050 con 0x7fabe806ab20 2023-11-04T05:28:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabee7fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80cd780 con 0x7fabe80cc7a0 2023-11-04T05:28:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabe77fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabe7fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabe806ab20 0x7fabe80728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabe77fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe80cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:42676/0 (socket says 172.21.15.3:42676) 2023-11-04T05:28:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabe77fe700 1 -- 172.21.15.3:0/2630471746 learned_addr learned my addr 172.21.15.3:0/2630471746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.513+0000 7fabecd98700 1 --1- 172.21.15.3:0/2630471746 >> v1:172.21.15.188:6789/0 conn(0x7fabe806af00 0x7fabe815ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:35510/0 (socket says 172.21.15.3:35510) 2023-11-04T05:28:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe7fff700 1 -- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 msgr2=0x7fabe80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe7fff700 1 --2- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe80cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe7fff700 1 -- 172.21.15.3:0/2630471746 >> v1:172.21.15.188:6789/0 conn(0x7fabe806af00 legacy=0x7fabe815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe7fff700 1 -- 172.21.15.3:0/2630471746 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80cda00 con 0x7fabe806ab20 2023-11-04T05:28:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe6ffd700 1 -- 172.21.15.3:0/2630471746 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459375516 0 0) 0x7fabe8072df0 con 0x7fabe806af00 2023-11-04T05:28:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe77fe700 1 --2- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe7fff700 1 --2- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806ab20 0x7fabe80728b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009840 tx=0x7fabd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb68a047d6468df3 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe6ffd700 1 -- 172.21.15.3:0/2630471746 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8011830 con 0x7fabe806ab20 2023-11-04T05:28:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe6ffd700 1 -- 172.21.15.3:0/2630471746 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd8011990 con 0x7fabe806ab20 2023-11-04T05:28:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.514+0000 7fabe6ffd700 1 -- 172.21.15.3:0/2630471746 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8011d20 con 0x7fabe806ab20 2023-11-04T05:28:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 -- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806ab20 msgr2=0x7fabe80728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806ab20 0x7fabe80728b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009840 tx=0x7fabd8009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 -- 172.21.15.3:0/2630471746 shutdown_connections 2023-11-04T05:28:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806ab20 0x7fabe80728b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2630471746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 -- 172.21.15.3:0/2630471746 >> 172.21.15.3:0/2630471746 conn(0x7fabe806df90 msgr2=0x7fabe806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 -- 172.21.15.3:0/2630471746 shutdown_connections 2023-11-04T05:28:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.515+0000 7fabee7fb700 1 -- 172.21.15.3:0/2630471746 wait complete. 2023-11-04T05:28:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.516+0000 7fabee7fb700 1 Processor -- start 2023-11-04T05:28:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.516+0000 7fabee7fb700 1 -- start start 2023-11-04T05:28:29.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabee7fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 0x7fabe8154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabee7fb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806af00 0x7fabe80c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabee7fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe81e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabee7fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80c6850 con 0x7fabe806ab20 2023-11-04T05:28:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabee7fb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe815b3f0 con 0x7fabe806af00 2023-11-04T05:28:29.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabee7fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe815b6d0 con 0x7fabe80cc7a0 2023-11-04T05:28:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 0x7fabe8154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 0x7fabe8154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51624/0 (socket says 172.21.15.3:51624) 2023-11-04T05:28:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 -- 172.21.15.3:0/2397491608 learned_addr learned my addr 172.21.15.3:0/2397491608 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabecd98700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe81e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe77fe700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806af00 0x7fabe80c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 -- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 msgr2=0x7fabe81e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe81e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 -- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806af00 msgr2=0x7fabe80c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806af00 0x7fabe80c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe7fff700 1 -- 172.21.15.3:0/2397491608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009500 con 0x7fabe806ab20 2023-11-04T05:28:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.517+0000 7fabe77fe700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806af00 0x7fabe80c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.518+0000 7fabe7fff700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 0x7fabe8154d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009b90 tx=0x7fabd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.518+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd80038a0 con 0x7fabe806ab20 2023-11-04T05:28:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.518+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd801e070 con 0x7fabe806ab20 2023-11-04T05:28:29.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.518+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd80094a0 con 0x7fabe806ab20 2023-11-04T05:28:29.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.518+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8003c40 con 0x7fabe806ab20 2023-11-04T05:28:29.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.518+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe815bc90 con 0x7fabe806ab20 2023-11-04T05:28:29.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.519+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe8072090 con 0x7fabe806ab20 2023-11-04T05:28:29.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.519+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabd802d450 con 0x7fabe806ab20 2023-11-04T05:28:29.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.519+0000 7fabe4ff9700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fabc806af30 0x7fabc806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:29.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.520+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fabd808a810 con 0x7fabe806ab20 2023-11-04T05:28:29.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.520+0000 7fabe77fe700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fabc806af30 0x7fabc806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:29.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.520+0000 7fabe77fe700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fabc806af30 0x7fabc806d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fabd0006fd0 tx=0x7fabd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:29.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.524+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fabe8072090 con 0x7fabe806ab20 2023-11-04T05:28:29.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.770+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fabe8072090 con 0x7fabe806ab20 2023-11-04T05:28:29.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.771+0000 7fabe4ff9700 1 -- 172.21.15.3:0/2397491608 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fabe8072090 con 0x7fabe806ab20 2023-11-04T05:28:29.773 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-04T05:28:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fabc806af30 msgr2=0x7fabc806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fabc806af30 0x7fabc806d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fabd0006fd0 tx=0x7fabd0008040 comp rx=0 tx=0).stop 2023-11-04T05:28:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 msgr2=0x7fabe8154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:29.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 0x7fabe8154d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009b90 tx=0x7fabd8004750 comp rx=0 tx=0).stop 2023-11-04T05:28:29.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 shutdown_connections 2023-11-04T05:28:29.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fabc806af30 0x7fabc806d3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabe80cc7a0 0x7fabe81e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe806ab20 0x7fabe8154d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 --2- 172.21.15.3:0/2397491608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabe806af00 0x7fabe80c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 >> 172.21.15.3:0/2397491608 conn(0x7fabe806df90 msgr2=0x7fabe815fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 shutdown_connections 2023-11-04T05:28:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:29.773+0000 7fabee7fb700 1 -- 172.21.15.3:0/2397491608 wait complete. 2023-11-04T05:28:29.787 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-04T05:28:29.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-04T05:28:29.790 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-11-04T05:28:29.790 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-04T05:28:29.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:29.791 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-04T05:28:29.791 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-04T05:28:29.791 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-04T05:28:29.792 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-04T05:28:29.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:29.792 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-11-04T05:28:29.810 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:28:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:28:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:28:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:28:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-11-04T05:28:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:28:29.812 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:28:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:29.812 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:28:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:29.812 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-04T05:28:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-04T05:28:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42396sec preferred_lft 42396sec 2023-11-04T05:28:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-11-04T05:28:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:28:29.813 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:28:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:29.814 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-04T05:28:29.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:29.815 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:29.816 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:29.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-11-04T05:28:29.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-11-04T05:28:29.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-11-04T05:28:29.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-04T05:28:29.817 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:29.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:29.974 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-04T05:28:30.003 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-11-04T05:28:30.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.005 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-11-04T05:28:30.077 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-11-04T05:28:30.077 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-04T05:28:30.077 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-04T05:28:30.077 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-04T05:28:30.078 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-04T05:28:30.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.079 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:30.079 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:30.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-04T05:28:30.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-04T05:28:30.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-04T05:28:30.080 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:30.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.324 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-11-04T05:28:30.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.343 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-11-04T05:28:30.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.400 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:30.401 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:30.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-04T05:28:30.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-04T05:28:30.402 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:30.496 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-04T05:28:30.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.497 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:30.498 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:30.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-04T05:28:30.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-04T05:28:30.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-04T05:28:30.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-04T05:28:30.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-04T05:28:30.500 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:30.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:30.708 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:30.708 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:30.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-11-04T05:28:30.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-11-04T05:28:30.709 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:30.819 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-04T05:28:30.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:28:30.820 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:30.840 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-04T05:28:30.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:28:30.841 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:30.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-11-04T05:28:30.964 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2023-11-04T05:28:31.022 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2023-11-04T05:28:31.023 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2023-11-04T05:28:31.023 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2023-11-04T05:28:31.023 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2023-11-04T05:28:31.023 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2023-11-04T05:28:31.023 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2023-11-04T05:28:31.024 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2023-11-04T05:28:31.024 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2023-11-04T05:28:31.024 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/systemd 2023-11-04T05:28:31.024 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2023-11-04T05:28:31.024 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2023-11-04T05:28:31.024 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/rdma 2023-11-04T05:28:31.025 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/perf_event 2023-11-04T05:28:31.025 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/blkio 2023-11-04T05:28:31.025 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-11-04T05:28:31.025 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/cpuset 2023-11-04T05:28:31.025 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/devices 2023-11-04T05:28:31.025 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/freezer 2023-11-04T05:28:31.026 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/pids 2023-11-04T05:28:31.026 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/memory 2023-11-04T05:28:31.026 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-11-04T05:28:31.026 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/hugetlb 2023-11-04T05:28:31.026 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2023-11-04T05:28:31.027 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2023-11-04T05:28:31.028 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2023-11-04T05:28:31.028 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2023-11-04T05:28:31.028 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2023-11-04T05:28:31.028 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2023-11-04T05:28:31.028 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2023-11-04T05:28:31.028 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2023-11-04T05:28:31.029 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2023-11-04T05:28:31.029 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2023-11-04T05:28:31.029 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2023-11-04T05:28:31.029 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-04T05:28:31.029 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-04T05:28:31.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:31.030 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-11-04T05:28:31.079 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-04T05:28:31.080 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-04T05:28:31.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-11-04T05:28:31.149 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2023-11-04T05:28:31.206 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2023-11-04T05:28:31.207 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2023-11-04T05:28:31.207 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2023-11-04T05:28:31.207 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2023-11-04T05:28:31.208 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2023-11-04T05:28:31.208 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2023-11-04T05:28:31.208 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2023-11-04T05:28:31.208 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2023-11-04T05:28:31.208 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/systemd 2023-11-04T05:28:31.209 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2023-11-04T05:28:31.209 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2023-11-04T05:28:31.209 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/rdma 2023-11-04T05:28:31.209 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/perf_event 2023-11-04T05:28:31.209 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/blkio 2023-11-04T05:28:31.210 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-11-04T05:28:31.210 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/cpuset 2023-11-04T05:28:31.210 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/devices 2023-11-04T05:28:31.210 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/freezer 2023-11-04T05:28:31.211 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/pids 2023-11-04T05:28:31.211 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/memory 2023-11-04T05:28:31.211 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-11-04T05:28:31.211 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup/hugetlb 2023-11-04T05:28:31.211 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2023-11-04T05:28:31.212 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2023-11-04T05:28:31.212 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2023-11-04T05:28:31.212 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2023-11-04T05:28:31.212 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2023-11-04T05:28:31.212 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2023-11-04T05:28:31.213 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2023-11-04T05:28:31.213 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2023-11-04T05:28:31.213 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2023-11-04T05:28:31.213 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2023-11-04T05:28:31.213 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2023-11-04T05:28:31.214 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2023-11-04T05:28:31.214 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2023-11-04T05:28:31.214 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2023-11-04T05:28:31.214 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2023-11-04T05:28:31.214 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2023-11-04T05:28:31.215 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2023-11-04T05:28:31.215 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-04T05:28:31.215 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-04T05:28:31.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:31.216 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-11-04T05:28:31.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:2023-11-04T05:28:31.222+0000 7fef47ec55c0 -1 init, newargv = 0x558d83ba1660 newargc=15 2023-11-04T05:28:31.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[110854]: starting ceph client 2023-11-04T05:28:31.231 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[110854]: starting fuse 2023-11-04T05:28:31.245 INFO:teuthology.orchestra.run.smithi003.stdout:47 2023-11-04T05:28:31.245 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-04T05:28:31.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2023-11-04T05:28:31.246 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-11-04T05:28:31.247 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-11-04T05:28:31.247 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-11-04T05:28:31.247 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-11-04T05:28:31.248 DEBUG:teuthology.orchestra.run.smithi003:> 2023-11-04T05:28:31.248 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-11-04T05:28:31.248 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-04T05:28:31.249 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-11-04T05:28:31.249 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-04T05:28:31.249 DEBUG:teuthology.orchestra.run.smithi003:> 2023-11-04T05:28:31.250 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-11-04T05:28:31.250 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-11-04T05:28:31.250 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-11-04T05:28:31.251 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-11-04T05:28:31.251 DEBUG:teuthology.orchestra.run.smithi003:> 2023-11-04T05:28:31.251 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-11-04T05:28:31.252 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-04T05:28:31.252 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-04T05:28:31.253 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-04T05:28:31.253 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-11-04T05:28:31.253 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-11-04T05:28:31.254 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-11-04T05:28:31.254 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-04T05:28:31.254 DEBUG:teuthology.orchestra.run.smithi003:> 2023-11-04T05:28:31.255 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.admin")) 2023-11-04T05:28:31.255 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-11-04T05:28:31.370 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.admin.110854.asok 2023-11-04T05:28:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-04T05:28:31.381 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.110854.asok 2023-11-04T05:28:31.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:31.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.110854.asok status 2023-11-04T05:28:31.543 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-11-04T05:28:31.543 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-11-04T05:28:31.543 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "826ae4205210243850fc3f7bc3f8aea0d1f5ced2", 2023-11-04T05:28:31.543 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev)", 2023-11-04T05:28:31.544 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "admin", 2023-11-04T05:28:31.544 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-11-04T05:28:31.544 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-04T05:28:31.544 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "110854", 2023-11-04T05:28:31.544 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-11-04T05:28:31.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:28:31.545 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-11-04T05:28:31.545 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-11-04T05:28:31.545 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4455, 2023-11-04T05:28:31.545 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-11-04T05:28:31.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-11-04T05:28:31.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-11-04T05:28:31.546 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 4455 2023-11-04T05:28:31.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:28:31.546 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-11-04T05:28:31.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-11-04T05:28:31.547 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-11-04T05:28:31.547 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 537036684 2023-11-04T05:28:31.547 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:28:31.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:28:31.548 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-11-04T05:28:31.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-11-04T05:28:31.548 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-11-04T05:28:31.548 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 537036684 2023-11-04T05:28:31.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:28:31.549 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.4455 192.168.0.1:0/537036684", 2023-11-04T05:28:31.549 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/537036684", 2023-11-04T05:28:31.549 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-11-04T05:28:31.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 7, 2023-11-04T05:28:31.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 26, 2023-11-04T05:28:31.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-11-04T05:28:31.550 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-11-04T05:28:31.550 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-11-04T05:28:31.550 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-11-04T05:28:31.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:31.553 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-11-04T05:28:31.553 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:31.571 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-11-04T05:28:31.571 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:31.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:31.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:31.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:31.639 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-04T05:28:31.747 INFO:teuthology.orchestra.run.smithi003.stdout:# file: . 2023-11-04T05:28:31.747 INFO:teuthology.orchestra.run.smithi003.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-04T05:28:31.748 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:31.748 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:28:31.748 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:28:31.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.931+0000 7f5f849a1700 1 Processor -- start 2023-11-04T05:28:31.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.931+0000 7f5f849a1700 1 -- start start 2023-11-04T05:28:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f849a1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f8015da70 0x7f5f800d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f849a1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f800cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f849a1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f800cb270 0x7f5f80161e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f849a1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f800cdd70 con 0x7f5f8015da70 2023-11-04T05:28:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f849a1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f800ceaa0 con 0x7f5f800cb270 2023-11-04T05:28:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f849a1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800ced80 con 0x7f5f8015de50 2023-11-04T05:28:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f8015da70 0x7f5f800d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5f8015da70 0x7f5f800d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51648/0 (socket says 172.21.15.3:51648) 2023-11-04T05:28:31.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/4009112211 learned_addr learned my addr 172.21.15.3:0/4009112211 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:31.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7effd700 1 --2- 172.21.15.3:0/4009112211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f800cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7ffff700 1 --2- 172.21.15.3:0/4009112211 >> v2:172.21.15.188:3300/0 conn(0x7f5f800cb270 0x7f5f80161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/4009112211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 msgr2=0x7f5f800cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:31.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 --2- 172.21.15.3:0/4009112211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f800cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/4009112211 >> v2:172.21.15.188:3300/0 conn(0x7f5f800cb270 msgr2=0x7f5f80161e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.932+0000 7f5f7f7fe700 1 --2- 172.21.15.3:0/4009112211 >> v2:172.21.15.188:3300/0 conn(0x7f5f800cb270 0x7f5f80161e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/4009112211 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80161910 con 0x7f5f8015da70 2023-11-04T05:28:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f7f7fe700 1 --2- 172.21.15.3:0/4009112211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f8015da70 0x7f5f800d0ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f800c3b80 tx=0x7f5f70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bf255e5962f06f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f7e7fc700 1 -- 172.21.15.3:0/4009112211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70005e30 con 0x7f5f8015da70 2023-11-04T05:28:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f7e7fc700 1 -- 172.21.15.3:0/4009112211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f70004030 con 0x7f5f8015da70 2023-11-04T05:28:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f7e7fc700 1 -- 172.21.15.3:0/4009112211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f700043e0 con 0x7f5f8015da70 2023-11-04T05:28:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 -- 172.21.15.3:0/4009112211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f8015da70 msgr2=0x7f5f800d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 --2- 172.21.15.3:0/4009112211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f8015da70 0x7f5f800d0ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f800c3b80 tx=0x7f5f70009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 -- 172.21.15.3:0/4009112211 shutdown_connections 2023-11-04T05:28:31.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 --2- 172.21.15.3:0/4009112211 >> v2:172.21.15.188:3300/0 conn(0x7f5f800cb270 0x7f5f80161e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 --2- 172.21.15.3:0/4009112211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f8015da70 0x7f5f800d0ea0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 --2- 172.21.15.3:0/4009112211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f800cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 -- 172.21.15.3:0/4009112211 >> 172.21.15.3:0/4009112211 conn(0x7f5f8006dde0 msgr2=0x7f5f8006e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 -- 172.21.15.3:0/4009112211 shutdown_connections 2023-11-04T05:28:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.933+0000 7f5f849a1700 1 -- 172.21.15.3:0/4009112211 wait complete. 2023-11-04T05:28:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.935+0000 7f5f849a1700 1 Processor -- start 2023-11-04T05:28:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.935+0000 7f5f849a1700 1 -- start start 2023-11-04T05:28:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 0x7f5f801f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f8015da70 0x7f5f8015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f8015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f80154480 con 0x7f5f800cb270 2023-11-04T05:28:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f80153870 con 0x7f5f8015da70 2023-11-04T05:28:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f80153b50 con 0x7f5f8015de50 2023-11-04T05:28:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 0x7f5f801f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 0x7f5f801f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51654/0 (socket says 172.21.15.3:51654) 2023-11-04T05:28:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/3429614045 learned_addr learned my addr 172.21.15.3:0/3429614045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7ffff700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f8015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 msgr2=0x7f5f8015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:31.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f8015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7effd700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f8015da70 0x7f5f8015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f8015da70 msgr2=0x7f5f8015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:31.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f8015da70 0x7f5f8015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:31.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 -- 172.21.15.3:0/3429614045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f70009500 con 0x7f5f800cb270 2023-11-04T05:28:31.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f7f7fe700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 0x7f5f801f31f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009130 tx=0x7f5f70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:31.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70003dd0 con 0x7f5f800cb270 2023-11-04T05:28:31.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f70011420 con 0x7f5f800cb270 2023-11-04T05:28:31.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f80153e30 con 0x7f5f800cb270 2023-11-04T05:28:31.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f700117d0 con 0x7f5f800cb270 2023-11-04T05:28:31.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.936+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f80153500 con 0x7f5f800cb270 2023-11-04T05:28:31.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.938+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f800afdb0 con 0x7f5f800cb270 2023-11-04T05:28:31.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.938+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f70011930 con 0x7f5f800cb270 2023-11-04T05:28:31.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.938+0000 7f5f77fff700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5f6006af00 0x7f5f6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:31.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.939+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f7008aae0 con 0x7f5f800cb270 2023-11-04T05:28:31.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.939+0000 7f5f7effd700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5f6006af00 0x7f5f6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:31.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.939+0000 7f5f7effd700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5f6006af00 0x7f5f6006d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f8015c110 tx=0x7f5f68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:31.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:31.943+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5f70055930 con 0x7f5f800cb270 2023-11-04T05:28:32.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.146+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f8004f730 con 0x7f5f800cb270 2023-11-04T05:28:32.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.147+0000 7f5f77fff700 1 -- 172.21.15.3:0/3429614045 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f5f70058f50 con 0x7f5f800cb270 2023-11-04T05:28:32.150 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:32.150 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.149+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5f6006af00 msgr2=0x7f5f6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.149+0000 7f5f849a1700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5f6006af00 0x7f5f6006d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f8015c110 tx=0x7f5f68008040 comp rx=0 tx=0).stop 2023-11-04T05:28:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.149+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 msgr2=0x7f5f801f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.149+0000 7f5f849a1700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 0x7f5f801f31f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009130 tx=0x7f5f70004750 comp rx=0 tx=0).stop 2023-11-04T05:28:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 shutdown_connections 2023-11-04T05:28:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5f6006af00 0x7f5f6006d3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f8015de50 0x7f5f8015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f800cb270 0x7f5f801f31f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 --2- 172.21.15.3:0/3429614045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f8015da70 0x7f5f8015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 >> 172.21.15.3:0/3429614045 conn(0x7f5f8006dde0 msgr2=0x7f5f800b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 shutdown_connections 2023-11-04T05:28:32.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.150+0000 7f5f849a1700 1 -- 172.21.15.3:0/3429614045 wait complete. 2023-11-04T05:28:32.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:28:32.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:28:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.293+0000 7f7a11c70700 1 Processor -- start 2023-11-04T05:28:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.293+0000 7f7a11c70700 1 -- start start 2023-11-04T05:28:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a11c70700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7a0c155df0 0x7f7a0c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a11c70700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 0x7f7a0c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a11c70700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c0b0f20 con 0x7f7a0c1601a0 2023-11-04T05:28:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a11c70700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c15f4a0 con 0x7f7a0c155df0 2023-11-04T05:28:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a11c70700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15c770 con 0x7f7a0c15be50 2023-11-04T05:28:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a0bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7a0c1601a0 0x7f7a0c06ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57014/0 (socket says 172.21.15.3:57014) 2023-11-04T05:28:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.294+0000 7f7a0bfff700 1 -- 172.21.15.3:0/3480173657 learned_addr learned my addr 172.21.15.3:0/3480173657 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0affd700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 0x7f7a0c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0b7fe700 1 --2- 172.21.15.3:0/3480173657 >> v2:172.21.15.188:3300/0 conn(0x7f7a0c155df0 0x7f7a0c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956248430 0 0) 0x7f7a04001490 con 0x7f7a0c1601a0 2023-11-04T05:28:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f80035b0 con 0x7f7a0c1601a0 2023-11-04T05:28:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486135662 0 0) 0x7f79f80035b0 con 0x7f7a0c1601a0 2023-11-04T05:28:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 msgr2=0x7f7a0c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 0x7f7a0c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 >> v2:172.21.15.188:3300/0 conn(0x7f7a0c155df0 msgr2=0x7f7a0c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 --2- 172.21.15.3:0/3480173657 >> v2:172.21.15.188:3300/0 conn(0x7f7a0c155df0 0x7f7a0c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c15ca50 con 0x7f7a0c1601a0 2023-11-04T05:28:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f7a04001b80 con 0x7f7a0c1601a0 2023-11-04T05:28:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 >> v1:172.21.15.3:6789/0 conn(0x7f7a0c1601a0 legacy=0x7f7a0c06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 0x7f79f8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8003f80 0x7f79f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8004d90 0x7f79f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15f4a0 con 0x7f79f8004d90 2023-11-04T05:28:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15c770 con 0x7f79f8003f80 2023-11-04T05:28:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f800ba10 con 0x7f7a0c15be50 2023-11-04T05:28:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8004d90 0x7f79f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0b7fe700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 0x7f79f8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0affd700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8003f80 0x7f79f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0bfff700 1 -- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 msgr2=0x7f79f8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c15be50 0x7f79f8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0bfff700 1 -- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8003f80 msgr2=0x7f79f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8003f80 0x7f79f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.295+0000 7f7a0bfff700 1 -- 172.21.15.3:0/3480173657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80035b0 con 0x7f79f8004d90 2023-11-04T05:28:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8004d90 0x7f79f8005210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c159aa0 tx=0x7f79f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8107dee2573bae server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400a4c0 con 0x7f79f8004d90 2023-11-04T05:28:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04012600 con 0x7f79f8004d90 2023-11-04T05:28:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a0a7fc700 1 -- 172.21.15.3:0/3480173657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040129b0 con 0x7f79f8004d90 2023-11-04T05:28:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a11c70700 1 -- 172.21.15.3:0/3480173657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8004d90 msgr2=0x7f79f8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a11c70700 1 --2- 172.21.15.3:0/3480173657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8004d90 0x7f79f8005210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c159aa0 tx=0x7f79f8005880 comp rx=0 tx=0).stop 2023-11-04T05:28:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a0bfff700 1 -- 172.21.15.3:0/3480173657 reap_dead start 2023-11-04T05:28:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a11c70700 1 -- 172.21.15.3:0/3480173657 shutdown_connections 2023-11-04T05:28:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a11c70700 1 -- 172.21.15.3:0/3480173657 >> 172.21.15.3:0/3480173657 conn(0x7f7a0c072560 msgr2=0x7f7a0c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a11c70700 1 -- 172.21.15.3:0/3480173657 shutdown_connections 2023-11-04T05:28:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.296+0000 7f7a11c70700 1 -- 172.21.15.3:0/3480173657 wait complete. 2023-11-04T05:28:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.298+0000 7f7a11c70700 1 Processor -- start 2023-11-04T05:28:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.298+0000 7f7a11c70700 1 -- start start 2023-11-04T05:28:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a11c70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8004d90 0x7f7a0c0ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a11c70700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c152c10 0x7f7a0c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a11c70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 0x7f7a0c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a11c70700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c156520 con 0x7f7a0c15a710 2023-11-04T05:28:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a11c70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15b0d0 con 0x7f79f8004d90 2023-11-04T05:28:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a11c70700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c153090 con 0x7f7a0c152c10 2023-11-04T05:28:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 0x7f7a0c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 0x7f7a0c15ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51678/0 (socket says 172.21.15.3:51678) 2023-11-04T05:28:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.299+0000 7f7a0bfff700 1 -- 172.21.15.3:0/793990774 learned_addr learned my addr 172.21.15.3:0/793990774 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0affd700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c152c10 0x7f7a0c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0b7fe700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8004d90 0x7f7a0c0ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0bfff700 1 -- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c152c10 msgr2=0x7f7a0c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c152c10 0x7f7a0c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0bfff700 1 -- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8004d90 msgr2=0x7f7a0c0ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8004d90 0x7f7a0c0ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0bfff700 1 -- 172.21.15.3:0/793990774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0afdc0 con 0x7f7a0c15a710 2023-11-04T05:28:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a0bfff700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 0x7f7a0c15ab90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a04001e40 tx=0x7f7a04001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04001490 con 0x7f7a0c15a710 2023-11-04T05:28:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04001b50 con 0x7f7a0c15a710 2023-11-04T05:28:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c0b00a0 con 0x7f7a0c15a710 2023-11-04T05:28:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0401e670 con 0x7f7a0c15a710 2023-11-04T05:28:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.300+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c156700 con 0x7f7a0c15a710 2023-11-04T05:28:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.301+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c156520 con 0x7f7a0c15a710 2023-11-04T05:28:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.301+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a0401e7d0 con 0x7f7a0c15a710 2023-11-04T05:28:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.302+0000 7f7a03fff700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f79ec06af00 0x7f79ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.302+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7a0408b360 con 0x7f7a0c15a710 2023-11-04T05:28:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.302+0000 7f7a0b7fe700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f79ec06af00 0x7f79ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.302+0000 7f7a0b7fe700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f79ec06af00 0x7f79ec06d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009c20 tx=0x7f79fc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.306+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7a040561b0 con 0x7f7a0c15a710 2023-11-04T05:28:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.510+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a0c0b0f20 con 0x7f7a0c15a710 2023-11-04T05:28:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.511+0000 7f7a03fff700 1 -- 172.21.15.3:0/793990774 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a0c0b0f20 con 0x7f7a0c15a710 2023-11-04T05:28:32.513 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:28:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.512+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f79ec06af00 msgr2=0x7f79ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.512+0000 7f7a11c70700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f79ec06af00 0x7f79ec06d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009c20 tx=0x7f79fc008040 comp rx=0 tx=0).stop 2023-11-04T05:28:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.512+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 msgr2=0x7f7a0c15ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.512+0000 7f7a11c70700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 0x7f7a0c15ab90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a04001e40 tx=0x7f7a04001360 comp rx=0 tx=0).stop 2023-11-04T05:28:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 shutdown_connections 2023-11-04T05:28:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f79ec06af00 0x7f79ec06d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f8004d90 0x7f7a0c0ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a0c15a710 0x7f7a0c15ab90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 --2- 172.21.15.3:0/793990774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a0c152c10 0x7f7a0c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 >> 172.21.15.3:0/793990774 conn(0x7f7a0c072560 msgr2=0x7f7a0c154880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:32.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 shutdown_connections 2023-11-04T05:28:32.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.513+0000 7f7a11c70700 1 -- 172.21.15.3:0/793990774 wait complete. 2023-11-04T05:28:32.527 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:28:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.653+0000 7f3a5696f700 1 Processor -- start 2023-11-04T05:28:32.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.653+0000 7f3a5696f700 1 -- start start 2023-11-04T05:28:32.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a5696f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a501611d0 0x7f3a500ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a5696f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3a500cd4b0 0x7f3a5006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a5696f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a5006b060 0x7f3a500aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a5696f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a500ce0a0 con 0x7f3a5006b060 2023-11-04T05:28:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a5696f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5006b4f0 con 0x7f3a500cd4b0 2023-11-04T05:28:32.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a5696f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500caca0 con 0x7f3a501611d0 2023-11-04T05:28:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a5006b060 0x7f3a500aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a5006b060 0x7f3a500aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51686/0 (socket says 172.21.15.3:51686) 2023-11-04T05:28:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 -- 172.21.15.3:0/3843985332 learned_addr learned my addr 172.21.15.3:0/3843985332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a4ffff700 1 --2- 172.21.15.3:0/3843985332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a501611d0 0x7f3a500ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a4f7fe700 1 --2- 172.21.15.3:0/3843985332 >> v2:172.21.15.188:3300/0 conn(0x7f3a500cd4b0 0x7f3a5006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 -- 172.21.15.3:0/3843985332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a501611d0 msgr2=0x7f3a500ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 --2- 172.21.15.3:0/3843985332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a501611d0 0x7f3a500ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 -- 172.21.15.3:0/3843985332 >> v2:172.21.15.188:3300/0 conn(0x7f3a500cd4b0 msgr2=0x7f3a5006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 --2- 172.21.15.3:0/3843985332 >> v2:172.21.15.188:3300/0 conn(0x7f3a500cd4b0 0x7f3a5006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 -- 172.21.15.3:0/3843985332 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500cfe40 con 0x7f3a5006b060 2023-11-04T05:28:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a4f7fe700 1 --2- 172.21.15.3:0/3843985332 >> v2:172.21.15.188:3300/0 conn(0x7f3a500cd4b0 0x7f3a5006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.654+0000 7f3a54f0c700 1 --2- 172.21.15.3:0/3843985332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5006b060 0x7f3a500aff20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a500730e0 tx=0x7f3a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b736480d3c928586 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a4effd700 1 -- 172.21.15.3:0/3843985332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800e070 con 0x7f3a5006b060 2023-11-04T05:28:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a4effd700 1 -- 172.21.15.3:0/3843985332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4800b9c0 con 0x7f3a5006b060 2023-11-04T05:28:32.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a4effd700 1 -- 172.21.15.3:0/3843985332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800bd70 con 0x7f3a5006b060 2023-11-04T05:28:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 -- 172.21.15.3:0/3843985332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5006b060 msgr2=0x7f3a500aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 --2- 172.21.15.3:0/3843985332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5006b060 0x7f3a500aff20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a500730e0 tx=0x7f3a4800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 -- 172.21.15.3:0/3843985332 shutdown_connections 2023-11-04T05:28:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 --2- 172.21.15.3:0/3843985332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5006b060 0x7f3a500aff20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 --2- 172.21.15.3:0/3843985332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a501611d0 0x7f3a500ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 --2- 172.21.15.3:0/3843985332 >> v2:172.21.15.188:3300/0 conn(0x7f3a500cd4b0 0x7f3a5006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 -- 172.21.15.3:0/3843985332 >> 172.21.15.3:0/3843985332 conn(0x7f3a50072560 msgr2=0x7f3a50099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:32.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 -- 172.21.15.3:0/3843985332 shutdown_connections 2023-11-04T05:28:32.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.655+0000 7f3a5696f700 1 -- 172.21.15.3:0/3843985332 wait complete. 2023-11-04T05:28:32.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.656+0000 7f3a5696f700 1 Processor -- start 2023-11-04T05:28:32.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 -- start start 2023-11-04T05:28:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a5006b060 0x7f3a5014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 0x7f3a500ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a501611d0 0x7f3a501f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500c7d50 con 0x7f3a500cd4b0 2023-11-04T05:28:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5014f080 con 0x7f3a501611d0 2023-11-04T05:28:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a5696f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500ca560 con 0x7f3a5006b060 2023-11-04T05:28:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 0x7f3a500ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.657+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 0x7f3a500ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51690/0 (socket says 172.21.15.3:51690) 2023-11-04T05:28:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 -- 172.21.15.3:0/20625537 learned_addr learned my addr 172.21.15.3:0/20625537 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4ffff700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a5006b060 0x7f3a5014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a54f0c700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a501611d0 0x7f3a501f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 -- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a5006b060 msgr2=0x7f3a5014eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a5006b060 0x7f3a5014eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 -- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a501611d0 msgr2=0x7f3a501f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a501611d0 0x7f3a501f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 -- 172.21.15.3:0/20625537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a48009b30 con 0x7f3a500cd4b0 2023-11-04T05:28:32.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4ffff700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a5006b060 0x7f3a5014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a54f0c700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a501611d0 0x7f3a501f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4f7fe700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 0x7f3a500ca020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c740 tx=0x7f3a3800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38011800 con 0x7f3a500cd4b0 2023-11-04T05:28:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38011960 con 0x7f3a500cd4b0 2023-11-04T05:28:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38011cd0 con 0x7f3a500cd4b0 2023-11-04T05:28:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a500c7a70 con 0x7f3a500cd4b0 2023-11-04T05:28:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.658+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a500c8e90 con 0x7f3a500cd4b0 2023-11-04T05:28:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.659+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a500c89f0 con 0x7f3a500cd4b0 2023-11-04T05:28:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.660+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a38021350 con 0x7f3a500cd4b0 2023-11-04T05:28:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.660+0000 7f3a4cff9700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3a3006af00 0x7f3a3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.660+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3a38089840 con 0x7f3a500cd4b0 2023-11-04T05:28:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.660+0000 7f3a4ffff700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3a3006af00 0x7f3a3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.661+0000 7f3a4ffff700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3a3006af00 0x7f3a3006d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a5006fec0 tx=0x7f3a40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.664+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f3a38054610 con 0x7f3a500cd4b0 2023-11-04T05:28:32.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.867+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a50074df0 con 0x7f3a500cd4b0 2023-11-04T05:28:32.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.867+0000 7f3a4cff9700 1 -- 172.21.15.3:0/20625537 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a38057c30 con 0x7f3a500cd4b0 2023-11-04T05:28:32.871 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:32.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3a3006af00 msgr2=0x7f3a3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3a3006af00 0x7f3a3006d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a5006fec0 tx=0x7f3a40008040 comp rx=0 tx=0).stop 2023-11-04T05:28:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 msgr2=0x7f3a500ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 0x7f3a500ca020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c740 tx=0x7f3a3800cb00 comp rx=0 tx=0).stop 2023-11-04T05:28:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 shutdown_connections 2023-11-04T05:28:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3a3006af00 0x7f3a3006d3c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a501611d0 0x7f3a501f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a5006b060 0x7f3a5014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 --2- 172.21.15.3:0/20625537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a500cd4b0 0x7f3a500ca020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:32.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.869+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 >> 172.21.15.3:0/20625537 conn(0x7f3a50072560 msgr2=0x7f3a5006ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:32.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.870+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 shutdown_connections 2023-11-04T05:28:32.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:32.870+0000 7f3a5696f700 1 -- 172.21.15.3:0/20625537 wait complete. 2023-11-04T05:28:32.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:28:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.010+0000 7f4839fc7700 1 Processor -- start 2023-11-04T05:28:33.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f4839fc7700 1 -- start start 2023-11-04T05:28:33.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f4839fc7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340cee80 0x7f483415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f4839fc7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f483415b260 0x7f48340caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f4839fc7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340ccc70 con 0x7f48340ceaa0 2023-11-04T05:28:33.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f4839fc7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483415baa0 con 0x7f483415b260 2023-11-04T05:28:33.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f4839fc7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340ccec0 con 0x7f48340cee80 2023-11-04T05:28:33.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f48337fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f48340ceaa0 0x7f48340cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57044/0 (socket says 172.21.15.3:57044) 2023-11-04T05:28:33.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.011+0000 7f48337fe700 1 -- 172.21.15.3:0/170664688 learned_addr learned my addr 172.21.15.3:0/170664688 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:33.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4832ffd700 1 --2- 172.21.15.3:0/170664688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340cee80 0x7f483415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4833fff700 1 --2- 172.21.15.3:0/170664688 >> v2:172.21.15.188:3300/0 conn(0x7f483415b260 0x7f48340caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f48327fc700 1 -- 172.21.15.3:0/170664688 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339504185 0 0) 0x7f48340ccc70 con 0x7f48340ceaa0 2023-11-04T05:28:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f48327fc700 1 -- 172.21.15.3:0/170664688 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48200035b0 con 0x7f48340ceaa0 2023-11-04T05:28:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4832ffd700 1 -- 172.21.15.3:0/170664688 >> v2:172.21.15.188:3300/0 conn(0x7f483415b260 msgr2=0x7f48340caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:33.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4832ffd700 1 --2- 172.21.15.3:0/170664688 >> v2:172.21.15.188:3300/0 conn(0x7f483415b260 0x7f48340caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4832ffd700 1 -- 172.21.15.3:0/170664688 >> v1:172.21.15.3:6789/0 conn(0x7f48340ceaa0 legacy=0x7f48340cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4832ffd700 1 -- 172.21.15.3:0/170664688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340cd1a0 con 0x7f48340cee80 2023-11-04T05:28:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f4832ffd700 1 --2- 172.21.15.3:0/170664688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340cee80 0x7f483415ad20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4824009840 tx=0x7f4824009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c261685809ad30 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f48327fc700 1 -- 172.21.15.3:0/170664688 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824011830 con 0x7f48340cee80 2023-11-04T05:28:33.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f48327fc700 1 -- 172.21.15.3:0/170664688 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824011990 con 0x7f48340cee80 2023-11-04T05:28:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.012+0000 7f48327fc700 1 -- 172.21.15.3:0/170664688 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824011d20 con 0x7f48340cee80 2023-11-04T05:28:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 -- 172.21.15.3:0/170664688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340cee80 msgr2=0x7f483415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 --2- 172.21.15.3:0/170664688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340cee80 0x7f483415ad20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4824009840 tx=0x7f4824009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 -- 172.21.15.3:0/170664688 shutdown_connections 2023-11-04T05:28:33.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 --2- 172.21.15.3:0/170664688 >> v2:172.21.15.188:3300/0 conn(0x7f483415b260 0x7f48340caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 --2- 172.21.15.3:0/170664688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340cee80 0x7f483415ad20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 -- 172.21.15.3:0/170664688 >> 172.21.15.3:0/170664688 conn(0x7f4834072560 msgr2=0x7f4834099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:33.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 -- 172.21.15.3:0/170664688 shutdown_connections 2023-11-04T05:28:33.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.013+0000 7f4839fc7700 1 -- 172.21.15.3:0/170664688 wait complete. 2023-11-04T05:28:33.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.014+0000 7f4839fc7700 1 Processor -- start 2023-11-04T05:28:33.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.014+0000 7f4839fc7700 1 -- start start 2023-11-04T05:28:33.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4839fc7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340ceaa0 0x7f483414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4839fc7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48340cee80 0x7f48340b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4839fc7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 0x7f48341f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4839fc7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483414f080 con 0x7f483415b260 2023-11-04T05:28:33.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4839fc7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340c78a0 con 0x7f48340cee80 2023-11-04T05:28:33.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4839fc7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340c6730 con 0x7f48340ceaa0 2023-11-04T05:28:33.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 0x7f48341f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 0x7f48341f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51706/0 (socket says 172.21.15.3:51706) 2023-11-04T05:28:33.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 -- 172.21.15.3:0/1472601703 learned_addr learned my addr 172.21.15.3:0/1472601703 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:33.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f48337fe700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340ceaa0 0x7f483414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4832ffd700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48340cee80 0x7f48340b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 -- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340ceaa0 msgr2=0x7f483414eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340ceaa0 0x7f483414eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 -- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48340cee80 msgr2=0x7f48340b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48340cee80 0x7f48340b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f4833fff700 1 -- 172.21.15.3:0/1472601703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48240094a0 con 0x7f483415b260 2023-11-04T05:28:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.015+0000 7f48337fe700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340ceaa0 0x7f483414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.016+0000 7f4833fff700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 0x7f48341f1170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f482c00efc0 tx=0x7f482c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.016+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c009ba0 con 0x7f483415b260 2023-11-04T05:28:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.016+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c00f040 con 0x7f483415b260 2023-11-04T05:28:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.016+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48340c6a10 con 0x7f483415b260 2023-11-04T05:28:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.016+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c014480 con 0x7f483415b260 2023-11-04T05:28:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.016+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48340c6370 con 0x7f483415b260 2023-11-04T05:28:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.017+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48340c6f30 con 0x7f483415b260 2023-11-04T05:28:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.017+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f482c0145e0 con 0x7f483415b260 2023-11-04T05:28:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.018+0000 7f482bfff700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f481406af00 0x7f481406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.018+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f482c08a390 con 0x7f483415b260 2023-11-04T05:28:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.018+0000 7f48337fe700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f481406af00 0x7f481406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.018+0000 7f48337fe700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f481406af00 0x7f481406d3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f481c0011e0 tx=0x7f481c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.022+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f482c0551e0 con 0x7f483415b260 2023-11-04T05:28:33.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.222+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4834074df0 con 0x7f483415b260 2023-11-04T05:28:33.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.222+0000 7f482bfff700 1 -- 172.21.15.3:0/1472601703 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f482c058800 con 0x7f483415b260 2023-11-04T05:28:33.225 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:28:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f481406af00 msgr2=0x7f481406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f481406af00 0x7f481406d3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f481c0011e0 tx=0x7f481c009040 comp rx=0 tx=0).stop 2023-11-04T05:28:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 msgr2=0x7f48341f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 0x7f48341f1170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f482c00efc0 tx=0x7f482c00c3a0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 shutdown_connections 2023-11-04T05:28:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f481406af00 0x7f481406d3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f483415b260 0x7f48341f1170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48340ceaa0 0x7f483414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 --2- 172.21.15.3:0/1472601703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48340cee80 0x7f48340b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 >> 172.21.15.3:0/1472601703 conn(0x7f4834072560 msgr2=0x7f483406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 shutdown_connections 2023-11-04T05:28:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.224+0000 7f4839fc7700 1 -- 172.21.15.3:0/1472601703 wait complete. 2023-11-04T05:28:33.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:28:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.364+0000 7f762e17e700 1 Processor -- start 2023-11-04T05:28:33.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.364+0000 7f762e17e700 1 -- start start 2023-11-04T05:28:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f762e17e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76281611d0 0x7f76280ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f762e17e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f762806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f762e17e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280caca0 con 0x7f762806b060 2023-11-04T05:28:33.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f762e17e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280ce0a0 con 0x7f76281611d0 2023-11-04T05:28:33.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f762e17e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762806b4f0 con 0x7f76280cd4b0 2023-11-04T05:28:33.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f76277fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76281611d0 0x7f76280ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f7627fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f762806b060 0x7f76280aff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57062/0 (socket says 172.21.15.3:57062) 2023-11-04T05:28:33.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.365+0000 7f7627fff700 1 -- 172.21.15.3:0/3017190785 learned_addr learned my addr 172.21.15.3:0/3017190785 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:33.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f7626ffd700 1 --2- 172.21.15.3:0/3017190785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f762806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76277fe700 1 -- 172.21.15.3:0/3017190785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 msgr2=0x7f762806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:33.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76277fe700 1 --2- 172.21.15.3:0/3017190785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f762806ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76277fe700 1 -- 172.21.15.3:0/3017190785 >> v1:172.21.15.3:6789/0 conn(0x7f762806b060 legacy=0x7f76280aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76277fe700 1 -- 172.21.15.3:0/3017190785 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280cfe40 con 0x7f76281611d0 2023-11-04T05:28:33.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76267fc700 1 -- 172.21.15.3:0/3017190785 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525546064 0 0) 0x7f76280caca0 con 0x7f762806b060 2023-11-04T05:28:33.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76277fe700 1 --2- 172.21.15.3:0/3017190785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76280ccf70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7618009840 tx=0x7f7618009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c2a2becf8e31ea server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76267fc700 1 -- 172.21.15.3:0/3017190785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618011830 con 0x7f76281611d0 2023-11-04T05:28:33.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76267fc700 1 -- 172.21.15.3:0/3017190785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618011990 con 0x7f76281611d0 2023-11-04T05:28:33.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.366+0000 7f76267fc700 1 -- 172.21.15.3:0/3017190785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618011d20 con 0x7f76281611d0 2023-11-04T05:28:33.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 -- 172.21.15.3:0/3017190785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 msgr2=0x7f76280ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 --2- 172.21.15.3:0/3017190785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76280ccf70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7618009840 tx=0x7f7618009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:33.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 -- 172.21.15.3:0/3017190785 shutdown_connections 2023-11-04T05:28:33.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 --2- 172.21.15.3:0/3017190785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76280ccf70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 --2- 172.21.15.3:0/3017190785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f762806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 -- 172.21.15.3:0/3017190785 >> 172.21.15.3:0/3017190785 conn(0x7f7628072560 msgr2=0x7f7628099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:33.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 -- 172.21.15.3:0/3017190785 shutdown_connections 2023-11-04T05:28:33.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.367+0000 7f762e17e700 1 -- 172.21.15.3:0/3017190785 wait complete. 2023-11-04T05:28:33.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.368+0000 7f762e17e700 1 Processor -- start 2023-11-04T05:28:33.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.368+0000 7f762e17e700 1 -- start start 2023-11-04T05:28:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f762e17e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f762806b060 0x7f762815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f762e17e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f76280c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f762e17e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76281f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f762e17e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76280b31b0 con 0x7f76281611d0 2023-11-04T05:28:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f762e17e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281f7220 con 0x7f762806b060 2023-11-04T05:28:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f762e17e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76281f7500 con 0x7f76280cd4b0 2023-11-04T05:28:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76281f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76281f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51720/0 (socket says 172.21.15.3:51720) 2023-11-04T05:28:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 -- 172.21.15.3:0/4058165127 learned_addr learned my addr 172.21.15.3:0/4058165127 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:33.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7626ffd700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f76280c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f76277fe700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f762806b060 0x7f762815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 -- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 msgr2=0x7f76280c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f76280c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 -- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f762806b060 msgr2=0x7f762815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f762806b060 0x7f762815af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 -- 172.21.15.3:0/4058165127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180094a0 con 0x7f76281611d0 2023-11-04T05:28:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7626ffd700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f76280c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.369+0000 7f7627fff700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76281f6780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7620002230 tx=0x7f762000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.370+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762000ac20 con 0x7f76281611d0 2023-11-04T05:28:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.370+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762000ada0 con 0x7f76281611d0 2023-11-04T05:28:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.370+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76280b3450 con 0x7f76281611d0 2023-11-04T05:28:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.370+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620007310 con 0x7f76281611d0 2023-11-04T05:28:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.370+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762815ff00 con 0x7f76281611d0 2023-11-04T05:28:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.371+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76280b3970 con 0x7f76281611d0 2023-11-04T05:28:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.371+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7620007470 con 0x7f76281611d0 2023-11-04T05:28:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.371+0000 7f761ffff700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f760806af00 0x7f760806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.372+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7620089720 con 0x7f76281611d0 2023-11-04T05:28:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.372+0000 7f76277fe700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f760806af00 0x7f760806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.372+0000 7f76277fe700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f760806af00 0x7f760806d3c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76180090b0 tx=0x7f7618008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.376+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f76280b3970 con 0x7f76281611d0 2023-11-04T05:28:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.576+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f76280b3970 con 0x7f76281611d0 2023-11-04T05:28:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.576+0000 7f761ffff700 1 -- 172.21.15.3:0/4058165127 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7620018020 con 0x7f76281611d0 2023-11-04T05:28:33.579 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f760806af00 msgr2=0x7f760806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f760806af00 0x7f760806d3c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76180090b0 tx=0x7f7618008030 comp rx=0 tx=0).stop 2023-11-04T05:28:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 msgr2=0x7f76281f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76281f6780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7620002230 tx=0x7f762000a060 comp rx=0 tx=0).stop 2023-11-04T05:28:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 shutdown_connections 2023-11-04T05:28:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f760806af00 0x7f760806d3c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f762806b060 0x7f762815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76281611d0 0x7f76281f6780 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 --2- 172.21.15.3:0/4058165127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76280cd4b0 0x7f76280c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.578+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 >> 172.21.15.3:0/4058165127 conn(0x7f7628072560 msgr2=0x7f76280cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.579+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 shutdown_connections 2023-11-04T05:28:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.579+0000 7f762e17e700 1 -- 172.21.15.3:0/4058165127 wait complete. 2023-11-04T05:28:33.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T05:28:33.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.717+0000 7f599b1bf700 1 Processor -- start 2023-11-04T05:28:33.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.717+0000 7f599b1bf700 1 -- start start 2023-11-04T05:28:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f599b1bf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59940c6c30 0x7f59940ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f599b1bf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940d1080 0x7f59940cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f599b1bf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59940c4830 con 0x7f59940c6850 2023-11-04T05:28:33.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f599b1bf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59940cc740 con 0x7f59940c6c30 2023-11-04T05:28:33.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f599b1bf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59940cc460 con 0x7f59940d1080 2023-11-04T05:28:33.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5993fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59940c6c30 0x7f59940ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5998f5b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59940c6850 0x7f59940cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57072/0 (socket says 172.21.15.3:57072) 2023-11-04T05:28:33.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5998f5b700 1 -- 172.21.15.3:0/3321811469 learned_addr learned my addr 172.21.15.3:0/3321811469 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:33.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f599975c700 1 --2- 172.21.15.3:0/3321811469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940d1080 0x7f59940cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5993fff700 1 -- 172.21.15.3:0/3321811469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940d1080 msgr2=0x7f59940cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:33.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5993fff700 1 --2- 172.21.15.3:0/3321811469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940d1080 0x7f59940cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5993fff700 1 -- 172.21.15.3:0/3321811469 >> v1:172.21.15.3:6789/0 conn(0x7f59940c6850 legacy=0x7f59940cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5993fff700 1 -- 172.21.15.3:0/3321811469 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940c41e0 con 0x7f59940c6c30 2023-11-04T05:28:33.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f59937fe700 1 -- 172.21.15.3:0/3321811469 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433337720 0 0) 0x7f59940c4830 con 0x7f59940c6850 2023-11-04T05:28:33.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.718+0000 7f5993fff700 1 --2- 172.21.15.3:0/3321811469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6c30 0x7f59940ccdc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5980009840 tx=0x7f5980009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d677e91cdaf6a5dd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f59937fe700 1 -- 172.21.15.3:0/3321811469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980011830 con 0x7f59940c6c30 2023-11-04T05:28:33.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f59937fe700 1 -- 172.21.15.3:0/3321811469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980011990 con 0x7f59940c6c30 2023-11-04T05:28:33.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f59937fe700 1 -- 172.21.15.3:0/3321811469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980011d20 con 0x7f59940c6c30 2023-11-04T05:28:33.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 -- 172.21.15.3:0/3321811469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6c30 msgr2=0x7f59940ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 --2- 172.21.15.3:0/3321811469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6c30 0x7f59940ccdc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5980009840 tx=0x7f5980009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:33.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 -- 172.21.15.3:0/3321811469 shutdown_connections 2023-11-04T05:28:33.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 --2- 172.21.15.3:0/3321811469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940d1080 0x7f59940cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 --2- 172.21.15.3:0/3321811469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6c30 0x7f59940ccdc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 -- 172.21.15.3:0/3321811469 >> 172.21.15.3:0/3321811469 conn(0x7f59940d0560 msgr2=0x7f59940d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:33.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 -- 172.21.15.3:0/3321811469 shutdown_connections 2023-11-04T05:28:33.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.719+0000 7f599b1bf700 1 -- 172.21.15.3:0/3321811469 wait complete. 2023-11-04T05:28:33.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.720+0000 7f599b1bf700 1 Processor -- start 2023-11-04T05:28:33.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.720+0000 7f599b1bf700 1 -- start start 2023-11-04T05:28:33.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f599b1bf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 0x7f5994157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f599b1bf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940c6c30 0x7f59940c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f599b1bf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59940d1080 0x7f59940c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f599b1bf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599415b420 con 0x7f59940c6850 2023-11-04T05:28:33.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f599b1bf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599415b700 con 0x7f59940d1080 2023-11-04T05:28:33.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f599b1bf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599415b9e0 con 0x7f59940c6c30 2023-11-04T05:28:33.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f5998f5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 0x7f5994157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f5998f5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 0x7f5994157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51730/0 (socket says 172.21.15.3:51730) 2023-11-04T05:28:33.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.721+0000 7f5998f5b700 1 -- 172.21.15.3:0/4176614931 learned_addr learned my addr 172.21.15.3:0/4176614931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:33.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5993fff700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940c6c30 0x7f59940c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f599975c700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59940d1080 0x7f59940c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5998f5b700 1 -- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940c6c30 msgr2=0x7f59940c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5998f5b700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940c6c30 0x7f59940c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5998f5b700 1 -- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59940d1080 msgr2=0x7f59940c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5998f5b700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59940d1080 0x7f59940c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5998f5b700 1 -- 172.21.15.3:0/4176614931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800094a0 con 0x7f59940c6850 2023-11-04T05:28:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5993fff700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940c6c30 0x7f59940c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:33.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f5998f5b700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 0x7f5994157a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59880010e0 tx=0x7f598800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59880074e0 con 0x7f59940c6850 2023-11-04T05:28:33.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5988007640 con 0x7f59940c6850 2023-11-04T05:28:33.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599415bcc0 con 0x7f59940c6850 2023-11-04T05:28:33.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988005a70 con 0x7f59940c6850 2023-11-04T05:28:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.722+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599415b0d0 con 0x7f59940c6850 2023-11-04T05:28:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.723+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5994074df0 con 0x7f59940c6850 2023-11-04T05:28:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.723+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5988005bd0 con 0x7f59940c6850 2023-11-04T05:28:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.724+0000 7f59917fa700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597806af00 0x7f597806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.724+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5988088d90 con 0x7f59940c6850 2023-11-04T05:28:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.724+0000 7f5993fff700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597806af00 0x7f597806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.725+0000 7f5993fff700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597806af00 0x7f597806d3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59800090b0 tx=0x7f5980008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.729+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5988053be0 con 0x7f59940c6850 2023-11-04T05:28:33.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.950+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f599406cf20 con 0x7f59940c6850 2023-11-04T05:28:33.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.950+0000 7f59917fa700 1 -- 172.21.15.3:0/4176614931 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5988017070 con 0x7f59940c6850 2023-11-04T05:28:33.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597806af00 msgr2=0x7f597806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597806af00 0x7f597806d3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59800090b0 tx=0x7f5980008030 comp rx=0 tx=0).stop 2023-11-04T05:28:33.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 msgr2=0x7f5994157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:33.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 0x7f5994157a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59880010e0 tx=0x7f598800cae0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 shutdown_connections 2023-11-04T05:28:33.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597806af00 0x7f597806d3c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59940d1080 0x7f59940c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940c6850 0x7f5994157a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 --2- 172.21.15.3:0/4176614931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59940c6c30 0x7f59940c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:33.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 >> 172.21.15.3:0/4176614931 conn(0x7f59940d0560 msgr2=0x7f5994159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:33.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 shutdown_connections 2023-11-04T05:28:33.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:33.952+0000 7f599b1bf700 1 -- 172.21.15.3:0/4176614931 wait complete. 2023-11-04T05:28:33.957 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T05:28:33.966 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:28:33.967 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:28:33.984 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:34.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.147+0000 7fd8f476e700 1 Processor -- start 2023-11-04T05:28:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.147+0000 7fd8f476e700 1 -- start start 2023-11-04T05:28:34.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f476e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8ec15a7e0 0x7fd8ec0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f476e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8ec0b2c00 0x7fd8ec0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f476e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0cba20 con 0x7fd8ec0cecf0 2023-11-04T05:28:34.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f476e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0ce8e0 con 0x7fd8ec15a7e0 2023-11-04T05:28:34.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f476e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0cbc70 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f250a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8ec15a7e0 0x7fd8ec0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f2d0b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd8ec0cecf0 0x7fd8ec071c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57092/0 (socket says 172.21.15.3:57092) 2023-11-04T05:28:34.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f2d0b700 1 -- 172.21.15.3:0/2801895315 learned_addr learned my addr 172.21.15.3:0/2801895315 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:34.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.148+0000 7fd8f1d09700 1 --2- 172.21.15.3:0/2801895315 >> v2:172.21.15.188:3300/0 conn(0x7fd8ec0b2c00 0x7fd8ec0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f250a700 1 -- 172.21.15.3:0/2801895315 >> v2:172.21.15.188:3300/0 conn(0x7fd8ec0b2c00 msgr2=0x7fd8ec0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:34.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f250a700 1 --2- 172.21.15.3:0/2801895315 >> v2:172.21.15.188:3300/0 conn(0x7fd8ec0b2c00 0x7fd8ec0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f250a700 1 -- 172.21.15.3:0/2801895315 >> v1:172.21.15.3:6789/0 conn(0x7fd8ec0cecf0 legacy=0x7fd8ec071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f1508700 1 -- 172.21.15.3:0/2801895315 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999503981 0 0) 0x7fd8ec0cba20 con 0x7fd8ec0cecf0 2023-11-04T05:28:34.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f250a700 1 -- 172.21.15.3:0/2801895315 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0cbf50 con 0x7fd8ec15a7e0 2023-11-04T05:28:34.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f1d09700 1 --2- 172.21.15.3:0/2801895315 >> v2:172.21.15.188:3300/0 conn(0x7fd8ec0b2c00 0x7fd8ec0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:34.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f250a700 1 --2- 172.21.15.3:0/2801895315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec15a7e0 0x7fd8ec0cb4e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0ca2d0 tx=0x7fd8dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a311cd423f30cf89 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f1508700 1 -- 172.21.15.3:0/2801895315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc011830 con 0x7fd8ec15a7e0 2023-11-04T05:28:34.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f1508700 1 -- 172.21.15.3:0/2801895315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc011990 con 0x7fd8ec15a7e0 2023-11-04T05:28:34.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f1508700 1 -- 172.21.15.3:0/2801895315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc011d20 con 0x7fd8ec15a7e0 2023-11-04T05:28:34.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f476e700 1 -- 172.21.15.3:0/2801895315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec15a7e0 msgr2=0x7fd8ec0cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.149+0000 7fd8f476e700 1 --2- 172.21.15.3:0/2801895315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec15a7e0 0x7fd8ec0cb4e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0ca2d0 tx=0x7fd8dc009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:34.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.150+0000 7fd8f476e700 1 -- 172.21.15.3:0/2801895315 shutdown_connections 2023-11-04T05:28:34.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.150+0000 7fd8f476e700 1 --2- 172.21.15.3:0/2801895315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec15a7e0 0x7fd8ec0cb4e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.150+0000 7fd8f476e700 1 --2- 172.21.15.3:0/2801895315 >> v2:172.21.15.188:3300/0 conn(0x7fd8ec0b2c00 0x7fd8ec0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.150+0000 7fd8f476e700 1 -- 172.21.15.3:0/2801895315 >> 172.21.15.3:0/2801895315 conn(0x7fd8ec0cfd50 msgr2=0x7fd8ec0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:34.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.150+0000 7fd8f476e700 1 -- 172.21.15.3:0/2801895315 shutdown_connections 2023-11-04T05:28:34.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.150+0000 7fd8f476e700 1 -- 172.21.15.3:0/2801895315 wait complete. 2023-11-04T05:28:34.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.151+0000 7fd8f476e700 1 Processor -- start 2023-11-04T05:28:34.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.151+0000 7fd8f476e700 1 -- start start 2023-11-04T05:28:34.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 0x7fd8ec0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8ec0cecf0 0x7fd8ec0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8ec15a7e0 0x7fd8ec0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b1750 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0c4250 con 0x7fd8ec0cecf0 2023-11-04T05:28:34.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0c3920 con 0x7fd8ec15a7e0 2023-11-04T05:28:34.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 0x7fd8ec0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 0x7fd8ec0b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51746/0 (socket says 172.21.15.3:51746) 2023-11-04T05:28:34.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 -- 172.21.15.3:0/1436354336 learned_addr learned my addr 172.21.15.3:0/1436354336 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:34.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f1d09700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8ec0cecf0 0x7fd8ec0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f2d0b700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8ec15a7e0 0x7fd8ec0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 -- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8ec15a7e0 msgr2=0x7fd8ec0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8ec15a7e0 0x7fd8ec0ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 -- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8ec0cecf0 msgr2=0x7fd8ec0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8ec0cecf0 0x7fd8ec0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 -- 172.21.15.3:0/1436354336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0094a0 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f1d09700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8ec0cecf0 0x7fd8ec0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:34.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f250a700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 0x7fd8ec0b1210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc014040 tx=0x7fd8dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0038a0 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc01e070 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc003c20 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec0c3c00 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.152+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec0c6030 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.153+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec074df0 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.154+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8dc02d450 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.154+0000 7fd8e2ffd700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8d006af00 0x7fd8d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.154+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8dc08a8c0 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.154+0000 7fd8f1d09700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8d006af00 0x7fd8d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.155+0000 7fd8f1d09700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8d006af00 0x7fd8d006d3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec1f6e70 tx=0x7fd8e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.159+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd8dc055710 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.358+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8ec156610 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.360+0000 7fd8e2ffd700 1 -- 172.21.15.3:0/1436354336 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7fd8dc058d30 con 0x7fd8ec0b2c00 2023-11-04T05:28:34.362 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:34.362 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:34.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8d006af00 msgr2=0x7fd8d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8d006af00 0x7fd8d006d3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec1f6e70 tx=0x7fd8e8008040 comp rx=0 tx=0).stop 2023-11-04T05:28:34.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 msgr2=0x7fd8ec0b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 0x7fd8ec0b1210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc014040 tx=0x7fd8dc004750 comp rx=0 tx=0).stop 2023-11-04T05:28:34.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 shutdown_connections 2023-11-04T05:28:34.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8d006af00 0x7fd8d006d3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8ec15a7e0 0x7fd8ec0ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8ec0b2c00 0x7fd8ec0b1210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 --2- 172.21.15.3:0/1436354336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8ec0cecf0 0x7fd8ec0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 >> 172.21.15.3:0/1436354336 conn(0x7fd8ec0cfd50 msgr2=0x7fd8ec0b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 shutdown_connections 2023-11-04T05:28:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.362+0000 7fd8f476e700 1 -- 172.21.15.3:0/1436354336 wait complete. 2023-11-04T05:28:34.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:28:34.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:28:34.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.503+0000 7f8f72d56700 1 Processor -- start 2023-11-04T05:28:34.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.503+0000 7f8f72d56700 1 -- start start 2023-11-04T05:28:34.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f72d56700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f6c1611d0 0x7f8f6c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f72d56700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 0x7f8f6c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f72d56700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0caca0 con 0x7f8f6c0cd4b0 2023-11-04T05:28:34.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f72d56700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ce0a0 con 0x7f8f6c1611d0 2023-11-04T05:28:34.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f72d56700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0cfe40 con 0x7f8f6c06b230 2023-11-04T05:28:34.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8f6c0cd4b0 0x7f8f6c06ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57104/0 (socket says 172.21.15.3:57104) 2023-11-04T05:28:34.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6bfff700 1 -- 172.21.15.3:0/2275841435 learned_addr learned my addr 172.21.15.3:0/2275841435 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:34.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f712f3700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 0x7f8f6c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f70af2700 1 --2- 172.21.15.3:0/2275841435 >> v2:172.21.15.188:3300/0 conn(0x7f8f6c1611d0 0x7f8f6c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651465175 0 0) 0x7f8f6c0caca0 con 0x7f8f6c0cd4b0 2023-11-04T05:28:34.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f5c0035b0 con 0x7f8f6c0cd4b0 2023-11-04T05:28:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232266927 0 0) 0x7f8f5c0035b0 con 0x7f8f6c0cd4b0 2023-11-04T05:28:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 msgr2=0x7f8f6c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:34.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 0x7f8f6c0d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 >> v2:172.21.15.188:3300/0 conn(0x7f8f6c1611d0 msgr2=0x7f8f6c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/2275841435 >> v2:172.21.15.188:3300/0 conn(0x7f8f6c1611d0 0x7f8f6c0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0cf850 con 0x7f8f6c0cd4b0 2023-11-04T05:28:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f8f58001670 con 0x7f8f6c0cd4b0 2023-11-04T05:28:34.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 >> v1:172.21.15.3:6789/0 conn(0x7f8f6c0cd4b0 legacy=0x7f8f6c06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 0x7f8f5c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f5c004d90 0x7f8f5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ce0a0 con 0x7f8f5c003f80 2023-11-04T05:28:34.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0cfe40 con 0x7f8f5c004d90 2023-11-04T05:28:34.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.504+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c00ba10 con 0x7f8f6c06b230 2023-11-04T05:28:34.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f70af2700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 0x7f8f5c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f712f3700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f5c004d90 0x7f8f5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 -- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 msgr2=0x7f8f5c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c06b230 0x7f8f5c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 -- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f5c004d90 msgr2=0x7f8f5c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f5c004d90 0x7f8f5c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 -- 172.21.15.3:0/2275841435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0035b0 con 0x7f8f5c003f80 2023-11-04T05:28:34.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f5c004850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c240 tx=0x7f8f5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29353de1280eb39b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800cd80 con 0x7f8f5c003f80 2023-11-04T05:28:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f58007ba0 con 0x7f8f5c003f80 2023-11-04T05:28:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.505+0000 7f8f6b7fe700 1 -- 172.21.15.3:0/2275841435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58005b80 con 0x7f8f5c003f80 2023-11-04T05:28:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f72d56700 1 -- 172.21.15.3:0/2275841435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 msgr2=0x7f8f5c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f72d56700 1 --2- 172.21.15.3:0/2275841435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f5c004850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c240 tx=0x7f8f5c005880 comp rx=0 tx=0).stop 2023-11-04T05:28:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f712f3700 1 -- 172.21.15.3:0/2275841435 reap_dead start 2023-11-04T05:28:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f72d56700 1 -- 172.21.15.3:0/2275841435 shutdown_connections 2023-11-04T05:28:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f72d56700 1 -- 172.21.15.3:0/2275841435 >> 172.21.15.3:0/2275841435 conn(0x7f8f6c072560 msgr2=0x7f8f6c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f72d56700 1 -- 172.21.15.3:0/2275841435 shutdown_connections 2023-11-04T05:28:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.506+0000 7f8f72d56700 1 -- 172.21.15.3:0/2275841435 wait complete. 2023-11-04T05:28:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.507+0000 7f8f72d56700 1 Processor -- start 2023-11-04T05:28:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.507+0000 7f8f72d56700 1 -- start start 2023-11-04T05:28:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f72d56700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f6c1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f72d56700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c160230 0x7f8f6c15a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f72d56700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c15ae10 0x7f8f6c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f72d56700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0c7050 con 0x7f8f5c003f80 2023-11-04T05:28:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f72d56700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1606b0 con 0x7f8f6c160230 2023-11-04T05:28:34.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f72d56700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c15fda0 con 0x7f8f6c15ae10 2023-11-04T05:28:34.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f6c1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f6c1f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51766/0 (socket says 172.21.15.3:51766) 2023-11-04T05:28:34.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 -- 172.21.15.3:0/385488913 learned_addr learned my addr 172.21.15.3:0/385488913 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:34.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c160230 0x7f8f6c15a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f712f3700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c15ae10 0x7f8f6c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 -- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c15ae10 msgr2=0x7f8f6c0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c15ae10 0x7f8f6c0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 -- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c160230 msgr2=0x7f8f6c15a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c160230 0x7f8f6c15a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f70af2700 1 -- 172.21.15.3:0/385488913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0c8380 con 0x7f8f5c003f80 2023-11-04T05:28:34.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.508+0000 7f8f712f3700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c15ae10 0x7f8f6c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:34.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.509+0000 7f8f70af2700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f6c1f2d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011ad0 tx=0x7f8f60011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.509+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000f8e0 con 0x7f8f5c003f80 2023-11-04T05:28:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.509+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000fa60 con 0x7f8f5c003f80 2023-11-04T05:28:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.509+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c0c8660 con 0x7f8f5c003f80 2023-11-04T05:28:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.509+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000fe10 con 0x7f8f5c003f80 2023-11-04T05:28:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.509+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c15f700 con 0x7f8f5c003f80 2023-11-04T05:28:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.510+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c06f850 con 0x7f8f5c003f80 2023-11-04T05:28:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.510+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f60014440 con 0x7f8f5c003f80 2023-11-04T05:28:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.510+0000 7f8f697fa700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8f5006af00 0x7f8f5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.511+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8f60089ff0 con 0x7f8f5c003f80 2023-11-04T05:28:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.511+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8f5006af00 0x7f8f5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.511+0000 7f8f6bfff700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8f5006af00 0x7f8f5006d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f58007d50 tx=0x7f8f5800a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.515+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8f60054e70 con 0x7f8f5c003f80 2023-11-04T05:28:34.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.716+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f6c06f570 con 0x7f8f5c003f80 2023-11-04T05:28:34.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.717+0000 7f8f697fa700 1 -- 172.21.15.3:0/385488913 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f600220a0 con 0x7f8f5c003f80 2023-11-04T05:28:34.722 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:28:34.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.718+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8f5006af00 msgr2=0x7f8f5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.718+0000 7f8f72d56700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8f5006af00 0x7f8f5006d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f58007d50 tx=0x7f8f5800a000 comp rx=0 tx=0).stop 2023-11-04T05:28:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.718+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 msgr2=0x7f8f6c1f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.718+0000 7f8f72d56700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f6c1f2d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011ad0 tx=0x7f8f60011de0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 shutdown_connections 2023-11-04T05:28:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8f5006af00 0x7f8f5006d3c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c15ae10 0x7f8f6c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f5c003f80 0x7f8f6c1f2d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 --2- 172.21.15.3:0/385488913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c160230 0x7f8f6c15a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 >> 172.21.15.3:0/385488913 conn(0x7f8f6c072560 msgr2=0x7f8f6c06ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 shutdown_connections 2023-11-04T05:28:34.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.719+0000 7f8f72d56700 1 -- 172.21.15.3:0/385488913 wait complete. 2023-11-04T05:28:34.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:28:34.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.859+0000 7fb014652700 1 Processor -- start 2023-11-04T05:28:34.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.859+0000 7fb014652700 1 -- start start 2023-11-04T05:28:34.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb014652700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb00c06afa0 0x7fb00c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb014652700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0caca0 con 0x7fb00c06ab20 2023-11-04T05:28:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb014652700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0cfe40 con 0x7fb00c1611d0 2023-11-04T05:28:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb014652700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0ce0a0 con 0x7fb00c06afa0 2023-11-04T05:28:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb011bed700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb00c06ab20 0x7fb00c0d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57124/0 (socket says 172.21.15.3:57124) 2023-11-04T05:28:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb011bed700 1 -- 172.21.15.3:0/3307043491 learned_addr learned my addr 172.21.15.3:0/3307043491 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb012bef700 1 --2- 172.21.15.3:0/3307043491 >> v2:172.21.15.188:3300/0 conn(0x7fb00c06afa0 0x7fb00c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb0113ec700 1 -- 172.21.15.3:0/3307043491 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496504407 0 0) 0x7fb00c0caca0 con 0x7fb00c06ab20 2023-11-04T05:28:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb0113ec700 1 -- 172.21.15.3:0/3307043491 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff80035b0 con 0x7fb00c06ab20 2023-11-04T05:28:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb012bef700 1 -- 172.21.15.3:0/3307043491 >> v1:172.21.15.188:6789/0 conn(0x7fb00c1611d0 legacy=0x7fb00c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb0113ec700 1 -- 172.21.15.3:0/3307043491 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727176691 0 0) 0x7faff80035b0 con 0x7fb00c06ab20 2023-11-04T05:28:34.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb012bef700 1 -- 172.21.15.3:0/3307043491 >> v1:172.21.15.3:6789/0 conn(0x7fb00c06ab20 legacy=0x7fb00c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.860+0000 7fb012bef700 1 -- 172.21.15.3:0/3307043491 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0cf850 con 0x7fb00c06afa0 2023-11-04T05:28:34.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb012bef700 1 --2- 172.21.15.3:0/3307043491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06afa0 0x7fb00c0d09e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb004009d90 tx=0x7fb00400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a96d75d7fec3f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb0113ec700 1 -- 172.21.15.3:0/3307043491 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400e070 con 0x7fb00c06afa0 2023-11-04T05:28:34.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb0113ec700 1 -- 172.21.15.3:0/3307043491 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004003a20 con 0x7fb00c06afa0 2023-11-04T05:28:34.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb0113ec700 1 -- 172.21.15.3:0/3307043491 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc001670 con 0x7fb00c06afa0 2023-11-04T05:28:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb014652700 1 -- 172.21.15.3:0/3307043491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06afa0 msgr2=0x7fb00c0d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb014652700 1 --2- 172.21.15.3:0/3307043491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06afa0 0x7fb00c0d09e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb004009d90 tx=0x7fb00400b330 comp rx=0 tx=0).stop 2023-11-04T05:28:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb014652700 1 -- 172.21.15.3:0/3307043491 shutdown_connections 2023-11-04T05:28:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb014652700 1 --2- 172.21.15.3:0/3307043491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06afa0 0x7fb00c0d09e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.861+0000 7fb014652700 1 -- 172.21.15.3:0/3307043491 >> 172.21.15.3:0/3307043491 conn(0x7fb00c072560 msgr2=0x7fb00c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.862+0000 7fb014652700 1 -- 172.21.15.3:0/3307043491 shutdown_connections 2023-11-04T05:28:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.862+0000 7fb014652700 1 -- 172.21.15.3:0/3307043491 wait complete. 2023-11-04T05:28:34.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.863+0000 7fb014652700 1 Processor -- start 2023-11-04T05:28:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.863+0000 7fb014652700 1 -- start start 2023-11-04T05:28:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb014652700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06ab20 0x7fb00c1f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb014652700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 0x7fb00c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb014652700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb00c1611d0 0x7fb00c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb014652700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0b4610 con 0x7fb00c06afa0 2023-11-04T05:28:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb014652700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1f7ca0 con 0x7fb00c06ab20 2023-11-04T05:28:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb014652700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1f4550 con 0x7fb00c1611d0 2023-11-04T05:28:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 0x7fb00c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 0x7fb00c0b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51782/0 (socket says 172.21.15.3:51782) 2023-11-04T05:28:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 -- 172.21.15.3:0/2712758257 learned_addr learned my addr 172.21.15.3:0/2712758257 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb012bef700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb00c1611d0 0x7fb00c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb0123ee700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06ab20 0x7fb00c1f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 -- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb00c1611d0 msgr2=0x7fb00c15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb00c1611d0 0x7fb00c15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 -- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06ab20 msgr2=0x7fb00c1f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06ab20 0x7fb00c1f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.864+0000 7fb011bed700 1 -- 172.21.15.3:0/2712758257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb004009a70 con 0x7fb00c06afa0 2023-11-04T05:28:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.865+0000 7fb011bed700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 0x7fb00c0b5370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faffc0011e0 tx=0x7faffc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.865+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc004c80 con 0x7fb00c06afa0 2023-11-04T05:28:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.865+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc004de0 con 0x7fb00c06afa0 2023-11-04T05:28:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.865+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c0b4890 con 0x7fb00c06afa0 2023-11-04T05:28:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.865+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc007310 con 0x7fb00c06afa0 2023-11-04T05:28:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.865+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c0b4e50 con 0x7fb00c06afa0 2023-11-04T05:28:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.866+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c0b5070 con 0x7fb00c06afa0 2023-11-04T05:28:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.866+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faffc007470 con 0x7fb00c06afa0 2023-11-04T05:28:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.867+0000 7fb002ffd700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faff006af00 0x7faff006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.867+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faffc089540 con 0x7fb00c06afa0 2023-11-04T05:28:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.867+0000 7fb0123ee700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faff006af00 0x7faff006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.867+0000 7fb0123ee700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faff006af00 0x7faff006d3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb008007fd0 tx=0x7fb008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:34.871+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7faffc054300 con 0x7fb00c06afa0 2023-11-04T05:28:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.075+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb00c0b61a0 con 0x7fb00c06afa0 2023-11-04T05:28:35.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.075+0000 7fb002ffd700 1 -- 172.21.15.3:0/2712758257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7faffc057920 con 0x7fb00c06afa0 2023-11-04T05:28:35.078 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:35.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faff006af00 msgr2=0x7faff006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faff006af00 0x7faff006d3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb008007fd0 tx=0x7fb008009040 comp rx=0 tx=0).stop 2023-11-04T05:28:35.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 msgr2=0x7fb00c0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 0x7fb00c0b5370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faffc0011e0 tx=0x7faffc00c9e0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 shutdown_connections 2023-11-04T05:28:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faff006af00 0x7faff006d3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb00c06ab20 0x7fb00c1f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb00c1611d0 0x7fb00c15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 --2- 172.21.15.3:0/2712758257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb00c06afa0 0x7fb00c0b5370 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 >> 172.21.15.3:0/2712758257 conn(0x7fb00c072560 msgr2=0x7fb00c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 shutdown_connections 2023-11-04T05:28:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.078+0000 7fb014652700 1 -- 172.21.15.3:0/2712758257 wait complete. 2023-11-04T05:28:35.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:28:35.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.218+0000 7f726a55e700 1 Processor -- start 2023-11-04T05:28:35.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.218+0000 7f726a55e700 1 -- start start 2023-11-04T05:28:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f726a55e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7264159f50 0x7f726415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f726a55e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72640cd640 0x7f726415e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f726a55e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726406dd80 con 0x7f726406f2d0 2023-11-04T05:28:35.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f726a55e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726406ed60 con 0x7f7264159f50 2023-11-04T05:28:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f726a55e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726406d770 con 0x7f72640cd640 2023-11-04T05:28:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f7263fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7264159f50 0x7f726415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f7268afb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f726406f2d0 0x7f726406ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57140/0 (socket says 172.21.15.3:57140) 2023-11-04T05:28:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f7268afb700 1 -- 172.21.15.3:0/3087311752 learned_addr learned my addr 172.21.15.3:0/3087311752 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:35.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.219+0000 7f72637fe700 1 --2- 172.21.15.3:0/3087311752 >> v2:172.21.15.188:3300/0 conn(0x7f72640cd640 0x7f726415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7263fff700 1 -- 172.21.15.3:0/3087311752 >> v2:172.21.15.188:3300/0 conn(0x7f72640cd640 msgr2=0x7f726415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7263fff700 1 --2- 172.21.15.3:0/3087311752 >> v2:172.21.15.188:3300/0 conn(0x7f72640cd640 0x7f726415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7263fff700 1 -- 172.21.15.3:0/3087311752 >> v1:172.21.15.3:6789/0 conn(0x7f726406f2d0 legacy=0x7f726406ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7263fff700 1 -- 172.21.15.3:0/3087311752 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726415e7b0 con 0x7f7264159f50 2023-11-04T05:28:35.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7262ffd700 1 -- 172.21.15.3:0/3087311752 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195205132 0 0) 0x7f726406dd80 con 0x7f726406f2d0 2023-11-04T05:28:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f72637fe700 1 --2- 172.21.15.3:0/3087311752 >> v2:172.21.15.188:3300/0 conn(0x7f72640cd640 0x7f726415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7263fff700 1 --2- 172.21.15.3:0/3087311752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7264159f50 0x7f726415dbe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f726406e4c0 tx=0x7f7254009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5965bc7ee6dedc02 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7262ffd700 1 -- 172.21.15.3:0/3087311752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254010800 con 0x7f7264159f50 2023-11-04T05:28:35.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7262ffd700 1 -- 172.21.15.3:0/3087311752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254010960 con 0x7f7264159f50 2023-11-04T05:28:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f7262ffd700 1 -- 172.21.15.3:0/3087311752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254010cf0 con 0x7f7264159f50 2023-11-04T05:28:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f726a55e700 1 -- 172.21.15.3:0/3087311752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7264159f50 msgr2=0x7f726415dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f726a55e700 1 --2- 172.21.15.3:0/3087311752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7264159f50 0x7f726415dbe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f726406e4c0 tx=0x7f7254009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f726a55e700 1 -- 172.21.15.3:0/3087311752 shutdown_connections 2023-11-04T05:28:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f726a55e700 1 --2- 172.21.15.3:0/3087311752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7264159f50 0x7f726415dbe0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f726a55e700 1 --2- 172.21.15.3:0/3087311752 >> v2:172.21.15.188:3300/0 conn(0x7f72640cd640 0x7f726415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.220+0000 7f726a55e700 1 -- 172.21.15.3:0/3087311752 >> 172.21.15.3:0/3087311752 conn(0x7f72640cc860 msgr2=0x7f72640ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.221+0000 7f726a55e700 1 -- 172.21.15.3:0/3087311752 shutdown_connections 2023-11-04T05:28:35.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.221+0000 7f726a55e700 1 -- 172.21.15.3:0/3087311752 wait complete. 2023-11-04T05:28:35.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.222+0000 7f726a55e700 1 Processor -- start 2023-11-04T05:28:35.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.222+0000 7f726a55e700 1 -- start start 2023-11-04T05:28:35.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f726a55e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 0x7f7264142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f726a55e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72640cd640 0x7f726415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f726a55e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7264159f50 0x7f72640ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f726a55e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72640ae550 con 0x7f726406f2d0 2023-11-04T05:28:35.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f726a55e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72640ad8d0 con 0x7f7264159f50 2023-11-04T05:28:35.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f726a55e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72640adbb0 con 0x7f72640cd640 2023-11-04T05:28:35.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 0x7f7264142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 0x7f7264142940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51794/0 (socket says 172.21.15.3:51794) 2023-11-04T05:28:35.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 -- 172.21.15.3:0/1469786731 learned_addr learned my addr 172.21.15.3:0/1469786731 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:35.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 -- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72640cd640 msgr2=0x7f726415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f72637fe700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72640cd640 0x7f726415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7268afb700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7264159f50 0x7f72640ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72640cd640 0x7f726415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 -- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7264159f50 msgr2=0x7f72640ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7264159f50 0x7f72640ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 -- 172.21.15.3:0/1469786731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540094a0 con 0x7f726406f2d0 2023-11-04T05:28:35.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7268afb700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7264159f50 0x7f72640ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:35.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7263fff700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 0x7f7264142940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7254000c00 tx=0x7f7254004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.223+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725401c040 con 0x7f726406f2d0 2023-11-04T05:28:35.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.224+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72540038b0 con 0x7f726406f2d0 2023-11-04T05:28:35.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.224+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254003c60 con 0x7f726406f2d0 2023-11-04T05:28:35.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.224+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72640ade90 con 0x7f726406f2d0 2023-11-04T05:28:35.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.224+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72641563b0 con 0x7f726406f2d0 2023-11-04T05:28:35.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.224+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72640ae7c0 con 0x7f726406f2d0 2023-11-04T05:28:35.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.225+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f725402c460 con 0x7f726406f2d0 2023-11-04T05:28:35.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.225+0000 7f7260ff9700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f725006af00 0x7f725006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.226+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f725408c8c0 con 0x7f726406f2d0 2023-11-04T05:28:35.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.226+0000 7f72637fe700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f725006af00 0x7f725006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.226+0000 7f72637fe700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f725006af00 0x7f725006d3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f726415d660 tx=0x7f7258008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.230+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7254057710 con 0x7f726406f2d0 2023-11-04T05:28:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.431+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7264074df0 con 0x7f726406f2d0 2023-11-04T05:28:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.431+0000 7f7260ff9700 1 -- 172.21.15.3:0/1469786731 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f725405ad30 con 0x7f726406f2d0 2023-11-04T05:28:35.433 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:28:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f725006af00 msgr2=0x7f725006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f725006af00 0x7f725006d3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f726415d660 tx=0x7f7258008040 comp rx=0 tx=0).stop 2023-11-04T05:28:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 msgr2=0x7f7264142940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 0x7f7264142940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7254000c00 tx=0x7f7254004750 comp rx=0 tx=0).stop 2023-11-04T05:28:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 shutdown_connections 2023-11-04T05:28:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f725006af00 0x7f725006d3c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7264159f50 0x7f72640ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f726406f2d0 0x7f7264142940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 --2- 172.21.15.3:0/1469786731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72640cd640 0x7f726415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 >> 172.21.15.3:0/1469786731 conn(0x7f72640cc860 msgr2=0x7f72640af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 shutdown_connections 2023-11-04T05:28:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.433+0000 7f726a55e700 1 -- 172.21.15.3:0/1469786731 wait complete. 2023-11-04T05:28:35.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:28:35.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.573+0000 7fbcc74e4700 1 Processor -- start 2023-11-04T05:28:35.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.573+0000 7fbcc74e4700 1 -- start start 2023-11-04T05:28:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc74e4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc0158360 0x7fbcc006f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc74e4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcc00668e0 0x7fbcc0066d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc74e4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc00cf7e0 con 0x7fbcc015eda0 2023-11-04T05:28:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc74e4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc00672d0 con 0x7fbcc00668e0 2023-11-04T05:28:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc74e4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015e910 con 0x7fbcc0158360 2023-11-04T05:28:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc5a81700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcc00668e0 0x7fbcc0066d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc5a81700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcc00668e0 0x7fbcc0066d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51806/0 (socket says 172.21.15.3:51806) 2023-11-04T05:28:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc5a81700 1 -- 172.21.15.3:0/1026027595 learned_addr learned my addr 172.21.15.3:0/1026027595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.574+0000 7fbcc5280700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc0158360 0x7fbcc006f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc5a81700 1 -- 172.21.15.3:0/1026027595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc0158360 msgr2=0x7fbcc006f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc5a81700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc0158360 0x7fbcc006f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc5a81700 1 -- 172.21.15.3:0/1026027595 >> v1:172.21.15.188:6789/0 conn(0x7fbcc015eda0 legacy=0x7fbcc00ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc5a81700 1 -- 172.21.15.3:0/1026027595 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00cf500 con 0x7fbcc00668e0 2023-11-04T05:28:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc5280700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc0158360 0x7fbcc006f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc5a81700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc00668e0 0x7fbcc0066d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009e50 tx=0x7fbcb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9542374e0aede9 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcb7fff700 1 -- 172.21.15.3:0/1026027595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb800e070 con 0x7fbcc00668e0 2023-11-04T05:28:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcb7fff700 1 -- 172.21.15.3:0/1026027595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb800b9c0 con 0x7fbcc00668e0 2023-11-04T05:28:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcb7fff700 1 -- 172.21.15.3:0/1026027595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb800bd70 con 0x7fbcc00668e0 2023-11-04T05:28:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc74e4700 1 -- 172.21.15.3:0/1026027595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc00668e0 msgr2=0x7fbcc0066d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.575+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc00668e0 0x7fbcc0066d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009e50 tx=0x7fbcb800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.576+0000 7fbcc74e4700 1 -- 172.21.15.3:0/1026027595 shutdown_connections 2023-11-04T05:28:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.576+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc00668e0 0x7fbcc0066d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.576+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/1026027595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc0158360 0x7fbcc006f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.576+0000 7fbcc74e4700 1 -- 172.21.15.3:0/1026027595 >> 172.21.15.3:0/1026027595 conn(0x7fbcc00cc860 msgr2=0x7fbcc00ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.576+0000 7fbcc74e4700 1 -- 172.21.15.3:0/1026027595 shutdown_connections 2023-11-04T05:28:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.576+0000 7fbcc74e4700 1 -- 172.21.15.3:0/1026027595 wait complete. 2023-11-04T05:28:35.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.577+0000 7fbcc74e4700 1 Processor -- start 2023-11-04T05:28:35.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.577+0000 7fbcc74e4700 1 -- start start 2023-11-04T05:28:35.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00668e0 0x7fbcc01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 0x7fbcc01f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc015eda0 0x7fbcc01f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0159a50 con 0x7fbcc0158360 2023-11-04T05:28:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01f3290 con 0x7fbcc015eda0 2023-11-04T05:28:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01f7910 con 0x7fbcc00668e0 2023-11-04T05:28:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 0x7fbcc01f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 0x7fbcc01f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51810/0 (socket says 172.21.15.3:51810) 2023-11-04T05:28:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 -- 172.21.15.3:0/2075539005 learned_addr learned my addr 172.21.15.3:0/2075539005 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc5280700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00668e0 0x7fbcc01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc5a81700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc015eda0 0x7fbcc01f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 -- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00668e0 msgr2=0x7fbcc01587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00668e0 0x7fbcc01587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 -- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc015eda0 msgr2=0x7fbcc01f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc015eda0 0x7fbcc01f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 -- 172.21.15.3:0/2075539005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8009b30 con 0x7fbcc0158360 2023-11-04T05:28:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc5280700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00668e0 0x7fbcc01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc5a81700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc015eda0 0x7fbcc01f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:35.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc4a7f700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 0x7fbcc01f2d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c450 tx=0x7fbcbc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0052d0 con 0x7fbcc0158360 2023-11-04T05:28:35.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc005430 con 0x7fbcc0158360 2023-11-04T05:28:35.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0038c0 con 0x7fbcc0158360 2023-11-04T05:28:35.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.578+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc01591b0 con 0x7fbcc0158360 2023-11-04T05:28:35.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.579+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc0159770 con 0x7fbcc0158360 2023-11-04T05:28:35.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.579+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc0074df0 con 0x7fbcc0158360 2023-11-04T05:28:35.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.580+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcbc003a20 con 0x7fbcc0158360 2023-11-04T05:28:35.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.580+0000 7fbcb5ffb700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbca406af00 0x7fbca406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.580+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbcbc088cb0 con 0x7fbcc0158360 2023-11-04T05:28:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.580+0000 7fbcc5280700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbca406af00 0x7fbca406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.581+0000 7fbcc5280700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbca406af00 0x7fbca406d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00099f0 tx=0x7fbcb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.585+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fbcbc053a70 con 0x7fbcc0158360 2023-11-04T05:28:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.783+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbcc015d5d0 con 0x7fbcc0158360 2023-11-04T05:28:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.784+0000 7fbcb5ffb700 1 -- 172.21.15.3:0/2075539005 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbcbc0170a0 con 0x7fbcc0158360 2023-11-04T05:28:35.786 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:35.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbca406af00 msgr2=0x7fbca406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbca406af00 0x7fbca406d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00099f0 tx=0x7fbcb0008040 comp rx=0 tx=0).stop 2023-11-04T05:28:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 msgr2=0x7fbcc01f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 0x7fbcc01f2d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c450 tx=0x7fbcbc00c810 comp rx=0 tx=0).stop 2023-11-04T05:28:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 shutdown_connections 2023-11-04T05:28:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbca406af00 0x7fbca406d3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc015eda0 0x7fbcc01f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00668e0 0x7fbcc01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 --2- 172.21.15.3:0/2075539005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcc0158360 0x7fbcc01f2d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 >> 172.21.15.3:0/2075539005 conn(0x7fbcc00cc860 msgr2=0x7fbcc0161b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:35.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 shutdown_connections 2023-11-04T05:28:35.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.786+0000 7fbcc74e4700 1 -- 172.21.15.3:0/2075539005 wait complete. 2023-11-04T05:28:35.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T05:28:35.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.924+0000 7f3d9709e700 1 Processor -- start 2023-11-04T05:28:35.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.924+0000 7f3d9709e700 1 -- start start 2023-11-04T05:28:35.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d9709e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d900c6850 0x7f3d900cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d9709e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3d900cd4d0 0x7f3d9015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d9709e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900c4830 con 0x7f3d900c6c30 2023-11-04T05:28:35.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d9709e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900cc740 con 0x7f3d900c6850 2023-11-04T05:28:35.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d9709e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900cc460 con 0x7f3d900cd4d0 2023-11-04T05:28:35.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d94e3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d900c6850 0x7f3d900cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d94e3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d900c6850 0x7f3d900cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51816/0 (socket says 172.21.15.3:51816) 2023-11-04T05:28:35.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d94e3a700 1 -- 172.21.15.3:0/2167600302 learned_addr learned my addr 172.21.15.3:0/2167600302 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:35.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.925+0000 7f3d9563b700 1 --2- 172.21.15.3:0/2167600302 >> v2:172.21.15.188:3300/0 conn(0x7f3d900cd4d0 0x7f3d9015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d94e3a700 1 -- 172.21.15.3:0/2167600302 >> v2:172.21.15.188:3300/0 conn(0x7f3d900cd4d0 msgr2=0x7f3d9015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:35.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d94e3a700 1 --2- 172.21.15.3:0/2167600302 >> v2:172.21.15.188:3300/0 conn(0x7f3d900cd4d0 0x7f3d9015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d94e3a700 1 -- 172.21.15.3:0/2167600302 >> v1:172.21.15.188:6789/0 conn(0x7f3d900c6c30 legacy=0x7f3d900ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d94e3a700 1 -- 172.21.15.3:0/2167600302 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900c41e0 con 0x7f3d900c6850 2023-11-04T05:28:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d94e3a700 1 --2- 172.21.15.3:0/2167600302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900cbc10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3d90072320 tx=0x7f3d84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75333f1f9a1f1e11 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d8f7fe700 1 -- 172.21.15.3:0/2167600302 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84005e30 con 0x7f3d900c6850 2023-11-04T05:28:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d8f7fe700 1 -- 172.21.15.3:0/2167600302 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84004030 con 0x7f3d900c6850 2023-11-04T05:28:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d8f7fe700 1 -- 172.21.15.3:0/2167600302 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840043e0 con 0x7f3d900c6850 2023-11-04T05:28:35.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d9709e700 1 -- 172.21.15.3:0/2167600302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 msgr2=0x7f3d900cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d9709e700 1 --2- 172.21.15.3:0/2167600302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900cbc10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3d90072320 tx=0x7f3d84009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d9709e700 1 -- 172.21.15.3:0/2167600302 shutdown_connections 2023-11-04T05:28:35.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d9709e700 1 --2- 172.21.15.3:0/2167600302 >> v2:172.21.15.188:3300/0 conn(0x7f3d900cd4d0 0x7f3d9015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d9709e700 1 --2- 172.21.15.3:0/2167600302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900cbc10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.926+0000 7f3d9709e700 1 -- 172.21.15.3:0/2167600302 >> 172.21.15.3:0/2167600302 conn(0x7f3d900d0560 msgr2=0x7f3d900d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:35.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.927+0000 7f3d9709e700 1 -- 172.21.15.3:0/2167600302 shutdown_connections 2023-11-04T05:28:35.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.927+0000 7f3d9709e700 1 -- 172.21.15.3:0/2167600302 wait complete. 2023-11-04T05:28:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.928+0000 7f3d9709e700 1 Processor -- start 2023-11-04T05:28:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.928+0000 7f3d9709e700 1 -- start start 2023-11-04T05:28:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d9709e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d9709e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d900c6c30 0x7f3d900b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d9709e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d900cd4d0 0x7f3d9014e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d9709e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900b0c90 con 0x7f3d900c6850 2023-11-04T05:28:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d9709e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9014d570 con 0x7f3d900cd4d0 2023-11-04T05:28:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d9709e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9014d850 con 0x7f3d900c6c30 2023-11-04T05:28:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d94e3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d94e3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51822/0 (socket says 172.21.15.3:51822) 2023-11-04T05:28:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.929+0000 7f3d94e3a700 1 -- 172.21.15.3:0/1384674332 learned_addr learned my addr 172.21.15.3:0/1384674332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d9563b700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d900cd4d0 0x7f3d9014e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d8ffff700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d900c6c30 0x7f3d900b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d94e3a700 1 -- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d900c6c30 msgr2=0x7f3d900b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d94e3a700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d900c6c30 0x7f3d900b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d94e3a700 1 -- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d900cd4d0 msgr2=0x7f3d9014e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d94e3a700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d900cd4d0 0x7f3d9014e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d94e3a700 1 -- 172.21.15.3:0/1384674332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84009500 con 0x7f3d900c6850 2023-11-04T05:28:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d8ffff700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d900c6c30 0x7f3d900b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d9563b700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d900cd4d0 0x7f3d9014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d94e3a700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900b1550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009c10 tx=0x7f3d84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400f8a0 con 0x7f3d900c6850 2023-11-04T05:28:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8400fa00 con 0x7f3d900c6850 2023-11-04T05:28:35.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9014db30 con 0x7f3d900c6850 2023-11-04T05:28:35.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400fdb0 con 0x7f3d900c6850 2023-11-04T05:28:35.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.930+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d90150180 con 0x7f3d900c6850 2023-11-04T05:28:35.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.931+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9014dd40 con 0x7f3d900c6850 2023-11-04T05:28:35.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.932+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d8402d430 con 0x7f3d900c6850 2023-11-04T05:28:35.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.932+0000 7f3d8d7fa700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3d7406af00 0x7f3d7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:35.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.932+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d8408c9e0 con 0x7f3d900c6850 2023-11-04T05:28:35.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.932+0000 7f3d8ffff700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3d7406af00 0x7f3d7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:35.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.933+0000 7f3d8ffff700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3d7406af00 0x7f3d7406d3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d900b2170 tx=0x7f3d7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:35.937+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f3d84057830 con 0x7f3d900c6850 2023-11-04T05:28:36.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.151+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d90074df0 con 0x7f3d900c6850 2023-11-04T05:28:36.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.152+0000 7f3d8d7fa700 1 -- 172.21.15.3:0/1384674332 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3d8405ae50 con 0x7f3d900c6850 2023-11-04T05:28:36.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3d7406af00 msgr2=0x7f3d7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:36.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3d7406af00 0x7f3d7406d3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d900b2170 tx=0x7f3d7c009040 comp rx=0 tx=0).stop 2023-11-04T05:28:36.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 msgr2=0x7f3d900b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900b1550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009c10 tx=0x7f3d84004750 comp rx=0 tx=0).stop 2023-11-04T05:28:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 shutdown_connections 2023-11-04T05:28:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f3d7406af00 0x7f3d7406d3c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d900cd4d0 0x7f3d9014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:36.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d900c6850 0x7f3d900b1550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 --2- 172.21.15.3:0/1384674332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d900c6c30 0x7f3d900b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 >> 172.21.15.3:0/1384674332 conn(0x7f3d900d0560 msgr2=0x7f3d9006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 shutdown_connections 2023-11-04T05:28:36.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:36.154+0000 7f3d9709e700 1 -- 172.21.15.3:0/1384674332 wait complete. 2023-11-04T05:28:36.159 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T05:28:36.168 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-11-04T05:28:36.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:36.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-04T05:28:36.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[110854]: fuse finished with error 0 and tester_r 0 2023-11-04T05:28:36.214 INFO:teuthology.orchestra.run:waiting for 300 2023-11-04T05:28:42.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:28:42.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:42.317 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-04T05:28:42.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.526+0000 7fd44e7b7700 1 Processor -- start 2023-11-04T05:28:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.526+0000 7fd44e7b7700 1 -- start start 2023-11-04T05:28:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44e7b7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 0x7fd4480d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44e7b7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 0x7fd44815b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44e7b7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44815af70 con 0x7fd4480cf330 2023-11-04T05:28:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44e7b7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4480cde10 con 0x7fd44815b400 2023-11-04T05:28:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44e7b7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd448072d30 con 0x7fd4480ccf50 2023-11-04T05:28:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44cd54700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 0x7fd44815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd4477fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 0x7fd4480d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44cd54700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 0x7fd44815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:50922/0 (socket says 172.21.15.3:50922) 2023-11-04T05:28:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd4477fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 0x7fd4480d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:42888/0 (socket says 172.21.15.3:42888) 2023-11-04T05:28:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd44cd54700 1 -- 172.21.15.3:0/2619770483 learned_addr learned my addr 172.21.15.3:0/2619770483 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd447fff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd4480cf330 0x7fd4480cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:35718/0 (socket says 172.21.15.3:35718) 2023-11-04T05:28:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.527+0000 7fd4477fe700 1 -- 172.21.15.3:0/2619770483 >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 msgr2=0x7fd44815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd4477fe700 1 --2- 172.21.15.3:0/2619770483 >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 0x7fd44815b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd4477fe700 1 -- 172.21.15.3:0/2619770483 >> v1:172.21.15.188:6789/0 conn(0x7fd4480cf330 legacy=0x7fd4480cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd4477fe700 1 -- 172.21.15.3:0/2619770483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480cd570 con 0x7fd4480ccf50 2023-11-04T05:28:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd44cd54700 1 --2- 172.21.15.3:0/2619770483 >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 0x7fd44815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd4477fe700 1 --2- 172.21.15.3:0/2619770483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 0x7fd4480d0a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd430009820 tx=0x7fd430009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a85bf459b8c8648 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd446ffd700 1 -- 172.21.15.3:0/2619770483 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430005e30 con 0x7fd4480ccf50 2023-11-04T05:28:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd446ffd700 1 -- 172.21.15.3:0/2619770483 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430006030 con 0x7fd4480ccf50 2023-11-04T05:28:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd446ffd700 1 -- 172.21.15.3:0/2619770483 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300063c0 con 0x7fd4480ccf50 2023-11-04T05:28:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd44e7b7700 1 -- 172.21.15.3:0/2619770483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 msgr2=0x7fd4480d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.528+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/2619770483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 0x7fd4480d0a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd430009820 tx=0x7fd430009b30 comp rx=0 tx=0).stop 2023-11-04T05:28:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.529+0000 7fd44e7b7700 1 -- 172.21.15.3:0/2619770483 shutdown_connections 2023-11-04T05:28:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.529+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/2619770483 >> v2:172.21.15.188:3300/0 conn(0x7fd44815b400 0x7fd44815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.529+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/2619770483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4480ccf50 0x7fd4480d0a40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.529+0000 7fd44e7b7700 1 -- 172.21.15.3:0/2619770483 >> 172.21.15.3:0/2619770483 conn(0x7fd44806df90 msgr2=0x7fd44806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:42.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.529+0000 7fd44e7b7700 1 -- 172.21.15.3:0/2619770483 shutdown_connections 2023-11-04T05:28:42.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.529+0000 7fd44e7b7700 1 -- 172.21.15.3:0/2619770483 wait complete. 2023-11-04T05:28:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.530+0000 7fd44e7b7700 1 Processor -- start 2023-11-04T05:28:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.530+0000 7fd44e7b7700 1 -- start start 2023-11-04T05:28:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44e7b7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 0x7fd4481596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44e7b7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4480cf330 0x7fd4480c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44e7b7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd44815b400 0x7fd44806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44e7b7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481f2520 con 0x7fd4480ccf50 2023-11-04T05:28:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44e7b7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480c9500 con 0x7fd4480cf330 2023-11-04T05:28:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44e7b7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480c97e0 con 0x7fd44815b400 2023-11-04T05:28:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 0x7fd4481596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 0x7fd4481596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51836/0 (socket says 172.21.15.3:51836) 2023-11-04T05:28:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 -- 172.21.15.3:0/1181487499 learned_addr learned my addr 172.21.15.3:0/1181487499 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd44cd54700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd44815b400 0x7fd44806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd4477fe700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4480cf330 0x7fd4480c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 -- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd44815b400 msgr2=0x7fd44806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd44815b400 0x7fd44806ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 -- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4480cf330 msgr2=0x7fd4480c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4480cf330 0x7fd4480c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 -- 172.21.15.3:0/1181487499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440009b30 con 0x7fd4480ccf50 2023-11-04T05:28:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd4477fe700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4480cf330 0x7fd4480c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.531+0000 7fd447fff700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 0x7fd4481596a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c520 tx=0x7fd43800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.532+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438005cb0 con 0x7fd4480ccf50 2023-11-04T05:28:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.532+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438005e10 con 0x7fd4480ccf50 2023-11-04T05:28:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.532+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4380072d0 con 0x7fd4480ccf50 2023-11-04T05:28:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.532+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd430009430 con 0x7fd4480ccf50 2023-11-04T05:28:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.532+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4480c9ac0 con 0x7fd4480ccf50 2023-11-04T05:28:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.533+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd448074df0 con 0x7fd4480ccf50 2023-11-04T05:28:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.533+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd438007430 con 0x7fd4480ccf50 2023-11-04T05:28:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.533+0000 7fd444ff9700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd42806af00 0x7fd42806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.534+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd438088cd0 con 0x7fd4480ccf50 2023-11-04T05:28:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.534+0000 7fd4477fe700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd42806af00 0x7fd42806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.534+0000 7fd4477fe700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd42806af00 0x7fd42806d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd430013040 tx=0x7fd430008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.538+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd438053b40 con 0x7fd4480ccf50 2023-11-04T05:28:42.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.741+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4480ca240 con 0x7fd4480ccf50 2023-11-04T05:28:42.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.743+0000 7fd444ff9700 1 -- 172.21.15.3:0/1181487499 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7fd438017020 con 0x7fd4480ccf50 2023-11-04T05:28:42.745 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:42.745 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd42806af00 msgr2=0x7fd42806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd42806af00 0x7fd42806d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd430013040 tx=0x7fd430008070 comp rx=0 tx=0).stop 2023-11-04T05:28:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 msgr2=0x7fd4481596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 0x7fd4481596a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c520 tx=0x7fd43800c8e0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 shutdown_connections 2023-11-04T05:28:42.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd42806af00 0x7fd42806d3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd44815b400 0x7fd44806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4480ccf50 0x7fd4481596a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 --2- 172.21.15.3:0/1181487499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4480cf330 0x7fd4480c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 >> 172.21.15.3:0/1181487499 conn(0x7fd44806df90 msgr2=0x7fd44815ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:42.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 shutdown_connections 2023-11-04T05:28:42.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.745+0000 7fd44e7b7700 1 -- 172.21.15.3:0/1181487499 wait complete. 2023-11-04T05:28:42.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:42.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.883+0000 7f5992e81700 1 Processor -- start 2023-11-04T05:28:42.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.883+0000 7f5992e81700 1 -- start start 2023-11-04T05:28:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f5992e81700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f598c0728b0 0x7f598c1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f5992e81700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c0cda00 con 0x7f598c0cc7a0 2023-11-04T05:28:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f5992e81700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c0cdce0 con 0x7f598c06ab20 2023-11-04T05:28:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f5992e81700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c0cf050 con 0x7f598c0728b0 2023-11-04T05:28:42.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f598bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f598c0cc7a0 0x7f598c06af00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57192/0 (socket says 172.21.15.3:57192) 2023-11-04T05:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f598bfff700 1 -- 172.21.15.3:0/4252065831 learned_addr learned my addr 172.21.15.3:0/4252065831 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f599141e700 1 --2- 172.21.15.3:0/4252065831 >> v2:172.21.15.188:3300/0 conn(0x7f598c0728b0 0x7f598c1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f598b7fe700 1 -- 172.21.15.3:0/4252065831 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655324239 0 0) 0x7f598c0cda00 con 0x7f598c0cc7a0 2023-11-04T05:28:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f598b7fe700 1 -- 172.21.15.3:0/4252065831 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f597c0035b0 con 0x7f598c0cc7a0 2023-11-04T05:28:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f599141e700 1 -- 172.21.15.3:0/4252065831 >> v1:172.21.15.188:6789/0 conn(0x7f598c06ab20 legacy=0x7f598c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f599141e700 1 -- 172.21.15.3:0/4252065831 >> v1:172.21.15.3:6789/0 conn(0x7f598c0cc7a0 legacy=0x7f598c06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.884+0000 7f599141e700 1 -- 172.21.15.3:0/4252065831 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0720b0 con 0x7f598c0728b0 2023-11-04T05:28:42.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f599141e700 1 --2- 172.21.15.3:0/4252065831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0728b0 0x7f598c1688e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5984009d90 tx=0x7f598400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=519b5667141f0914 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f598b7fe700 1 -- 172.21.15.3:0/4252065831 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598400e070 con 0x7f598c0728b0 2023-11-04T05:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f598b7fe700 1 -- 172.21.15.3:0/4252065831 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598400bb20 con 0x7f598c0728b0 2023-11-04T05:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f598b7fe700 1 -- 172.21.15.3:0/4252065831 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984012430 con 0x7f598c0728b0 2023-11-04T05:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f5992e81700 1 -- 172.21.15.3:0/4252065831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0728b0 msgr2=0x7f598c1688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f5992e81700 1 --2- 172.21.15.3:0/4252065831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0728b0 0x7f598c1688e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5984009d90 tx=0x7f598400b330 comp rx=0 tx=0).stop 2023-11-04T05:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f5992e81700 1 -- 172.21.15.3:0/4252065831 shutdown_connections 2023-11-04T05:28:42.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f5992e81700 1 --2- 172.21.15.3:0/4252065831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0728b0 0x7f598c1688e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.885+0000 7f5992e81700 1 -- 172.21.15.3:0/4252065831 >> 172.21.15.3:0/4252065831 conn(0x7f598c06df90 msgr2=0x7f598c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:42.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.886+0000 7f5992e81700 1 -- 172.21.15.3:0/4252065831 shutdown_connections 2023-11-04T05:28:42.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.886+0000 7f5992e81700 1 -- 172.21.15.3:0/4252065831 wait complete. 2023-11-04T05:28:42.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.887+0000 7f5992e81700 1 Processor -- start 2023-11-04T05:28:42.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.887+0000 7f5992e81700 1 -- start start 2023-11-04T05:28:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5992e81700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 0x7f598c15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5992e81700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f598c0728b0 0x7f598c0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5992e81700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0cc7a0 0x7f598c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5992e81700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0c8560 con 0x7f598c06ab20 2023-11-04T05:28:42.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5992e81700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c153f50 con 0x7f598c0cc7a0 2023-11-04T05:28:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5992e81700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0c8790 con 0x7f598c0728b0 2023-11-04T05:28:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 0x7f598c15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 0x7f598c15bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51848/0 (socket says 172.21.15.3:51848) 2023-11-04T05:28:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 -- 172.21.15.3:0/1918061259 learned_addr learned my addr 172.21.15.3:0/1918061259 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f598bfff700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f598c0728b0 0x7f598c0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f599141e700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0cc7a0 0x7f598c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 -- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f598c0728b0 msgr2=0x7f598c0c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f598c0728b0 0x7f598c0c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 -- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0cc7a0 msgr2=0x7f598c153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0cc7a0 0x7f598c153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 -- 172.21.15.3:0/1918061259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009a70 con 0x7f598c06ab20 2023-11-04T05:28:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f598bfff700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f598c0728b0 0x7f598c0c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f599141e700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0cc7a0 0x7f598c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f5990c1d700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 0x7f598c15bb90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f598000c4e0 tx=0x7f598000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.888+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800052d0 con 0x7f598c06ab20 2023-11-04T05:28:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.889+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980005430 con 0x7f598c06ab20 2023-11-04T05:28:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.889+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c0c8a70 con 0x7f598c06ab20 2023-11-04T05:28:42.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.889+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800038c0 con 0x7f598c06ab20 2023-11-04T05:28:42.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.889+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c0c9030 con 0x7f598c06ab20 2023-11-04T05:28:42.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.890+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c074df0 con 0x7f598c06ab20 2023-11-04T05:28:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.890+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5980003a20 con 0x7f598c06ab20 2023-11-04T05:28:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.890+0000 7f59897fa700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597006af00 0x7f597006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.890+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5980088e60 con 0x7f598c06ab20 2023-11-04T05:28:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.891+0000 7f598bfff700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597006af00 0x7f597006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.891+0000 7f598bfff700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597006af00 0x7f597006d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f598c15d750 tx=0x7f5978009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:42.895+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5980053ba0 con 0x7f598c06ab20 2023-11-04T05:28:43.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.099+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f598c0cd020 con 0x7f598c06ab20 2023-11-04T05:28:43.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.100+0000 7f59897fa700 1 -- 172.21.15.3:0/1918061259 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f59800571c0 con 0x7f598c06ab20 2023-11-04T05:28:43.103 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:43.103 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:43.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597006af00 msgr2=0x7f597006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597006af00 0x7f597006d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f598c15d750 tx=0x7f5978009040 comp rx=0 tx=0).stop 2023-11-04T05:28:43.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 msgr2=0x7f598c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 0x7f598c15bb90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f598000c4e0 tx=0x7f598000c8a0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 shutdown_connections 2023-11-04T05:28:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f597006af00 0x7f597006d3c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f598c0cc7a0 0x7f598c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f598c06ab20 0x7f598c15bb90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 --2- 172.21.15.3:0/1918061259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f598c0728b0 0x7f598c0c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 >> 172.21.15.3:0/1918061259 conn(0x7f598c06df90 msgr2=0x7f598c0caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 shutdown_connections 2023-11-04T05:28:43.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.103+0000 7f5992e81700 1 -- 172.21.15.3:0/1918061259 wait complete. 2023-11-04T05:28:43.118 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-04T05:28:43.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.244+0000 7f8906889700 1 Processor -- start 2023-11-04T05:28:43.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.244+0000 7f8906889700 1 -- start start 2023-11-04T05:28:43.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8906889700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f890015ba10 0x7f89000c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8906889700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89000b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8906889700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89000d11f0 0x7f89000b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8906889700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8900070a90 con 0x7f89000d11f0 2023-11-04T05:28:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8906889700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89000718f0 con 0x7f890015ba10 2023-11-04T05:28:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8906889700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89000c5720 con 0x7f89000d0d70 2023-11-04T05:28:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8904e26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89000d11f0 0x7f89000b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8904e26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89000d11f0 0x7f89000b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51860/0 (socket says 172.21.15.3:51860) 2023-11-04T05:28:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.245+0000 7f8904e26700 1 -- 172.21.15.3:0/275985158 learned_addr learned my addr 172.21.15.3:0/275985158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f88ff7fe700 1 --2- 172.21.15.3:0/275985158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89000b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f88fffff700 1 --2- 172.21.15.3:0/275985158 >> v2:172.21.15.188:3300/0 conn(0x7f890015ba10 0x7f89000c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8904e26700 1 -- 172.21.15.3:0/275985158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 msgr2=0x7f89000b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8904e26700 1 --2- 172.21.15.3:0/275985158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89000b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8904e26700 1 -- 172.21.15.3:0/275985158 >> v2:172.21.15.188:3300/0 conn(0x7f890015ba10 msgr2=0x7f89000c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8904e26700 1 --2- 172.21.15.3:0/275985158 >> v2:172.21.15.188:3300/0 conn(0x7f890015ba10 0x7f89000c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8904e26700 1 -- 172.21.15.3:0/275985158 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8900070ce0 con 0x7f89000d11f0 2023-11-04T05:28:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f88fffff700 1 --2- 172.21.15.3:0/275985158 >> v2:172.21.15.188:3300/0 conn(0x7f890015ba10 0x7f89000c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8904e26700 1 --2- 172.21.15.3:0/275985158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000d11f0 0x7f89000b3650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f89000ce0e0 tx=0x7f88f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8098c30ed7fb1d24 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f88feffd700 1 -- 172.21.15.3:0/275985158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800e070 con 0x7f89000d11f0 2023-11-04T05:28:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f88feffd700 1 -- 172.21.15.3:0/275985158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800b9c0 con 0x7f89000d11f0 2023-11-04T05:28:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f88feffd700 1 -- 172.21.15.3:0/275985158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800bd70 con 0x7f89000d11f0 2023-11-04T05:28:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8906889700 1 -- 172.21.15.3:0/275985158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000d11f0 msgr2=0x7f89000b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.246+0000 7f8906889700 1 --2- 172.21.15.3:0/275985158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000d11f0 0x7f89000b3650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f89000ce0e0 tx=0x7f88f800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 -- 172.21.15.3:0/275985158 shutdown_connections 2023-11-04T05:28:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 --2- 172.21.15.3:0/275985158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000d11f0 0x7f89000b3650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 --2- 172.21.15.3:0/275985158 >> v2:172.21.15.188:3300/0 conn(0x7f890015ba10 0x7f89000c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 --2- 172.21.15.3:0/275985158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89000b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 -- 172.21.15.3:0/275985158 >> 172.21.15.3:0/275985158 conn(0x7f89000ce110 msgr2=0x7f89000ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 -- 172.21.15.3:0/275985158 shutdown_connections 2023-11-04T05:28:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.247+0000 7f8906889700 1 -- 172.21.15.3:0/275985158 wait complete. 2023-11-04T05:28:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.248+0000 7f8906889700 1 Processor -- start 2023-11-04T05:28:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.248+0000 7f8906889700 1 -- start start 2023-11-04T05:28:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8906889700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89001569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8906889700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f890015ba10 0x7f89000b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8906889700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 0x7f89000cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8906889700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900156f00 con 0x7f89000b6f90 2023-11-04T05:28:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8906889700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89000b4770 con 0x7f890015ba10 2023-11-04T05:28:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8906889700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89001f4990 con 0x7f89000d0d70 2023-11-04T05:28:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 0x7f89000cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 0x7f89000cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51866/0 (socket says 172.21.15.3:51866) 2023-11-04T05:28:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 -- 172.21.15.3:0/974121225 learned_addr learned my addr 172.21.15.3:0/974121225 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f88fffff700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89001569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f88ff7fe700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f890015ba10 0x7f89000b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 -- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 msgr2=0x7f89001569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89001569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 -- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f890015ba10 msgr2=0x7f89000b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f890015ba10 0x7f89000b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 -- 172.21.15.3:0/974121225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8009b30 con 0x7f89000b6f90 2023-11-04T05:28:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.249+0000 7f8904e26700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 0x7f89000cf770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f88f8012000 tx=0x7f88f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.250+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f80099b0 con 0x7f89000b6f90 2023-11-04T05:28:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.250+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f8004250 con 0x7f89000b6f90 2023-11-04T05:28:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.250+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89001f4c70 con 0x7f89000b6f90 2023-11-04T05:28:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.250+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8004600 con 0x7f89000b6f90 2023-11-04T05:28:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.250+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89001f5230 con 0x7f89000b6f90 2023-11-04T05:28:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.250+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89000c95d0 con 0x7f89000b6f90 2023-11-04T05:28:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.251+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88f8004760 con 0x7f89000b6f90 2023-11-04T05:28:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.251+0000 7f88fcff9700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f88e006af00 0x7f88e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.252+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f88f808ac60 con 0x7f89000b6f90 2023-11-04T05:28:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.252+0000 7f88fffff700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f88e006af00 0x7f88e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.252+0000 7f88fffff700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f88e006af00 0x7f88e006d3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88f0006fd0 tx=0x7f88f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.256+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f89000c95d0 con 0x7f89000b6f90 2023-11-04T05:28:43.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.504+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89000c95d0 con 0x7f89000b6f90 2023-11-04T05:28:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.884+0000 7f88fcff9700 1 -- 172.21.15.3:0/974121225 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89000c95d0 con 0x7f89000b6f90 2023-11-04T05:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f88e006af00 msgr2=0x7f88e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f88e006af00 0x7f88e006d3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88f0006fd0 tx=0x7f88f0008040 comp rx=0 tx=0).stop 2023-11-04T05:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 msgr2=0x7f89000cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 0x7f89000cf770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f88f8012000 tx=0x7f88f8004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 shutdown_connections 2023-11-04T05:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f88e006af00 0x7f88e006d3c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89000b6f90 0x7f89000cf770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89000d0d70 0x7f89001569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 --2- 172.21.15.3:0/974121225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f890015ba10 0x7f89000b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.887+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 >> 172.21.15.3:0/974121225 conn(0x7f89000ce110 msgr2=0x7f89000c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.888+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 shutdown_connections 2023-11-04T05:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:43.888+0000 7f8906889700 1 -- 172.21.15.3:0/974121225 wait complete. 2023-11-04T05:28:43.902 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-04T05:28:43.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-04T05:28:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.031+0000 7f2dcd2c0700 1 Processor -- start 2023-11-04T05:28:44.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.031+0000 7f2dcd2c0700 1 -- start start 2023-11-04T05:28:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dcd2c0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2dc80b3820 0x7f2dc815f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dcd2c0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc8070a90 con 0x7f2dc80d0d70 2023-11-04T05:28:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dcd2c0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc80718f0 con 0x7f2dc815ba10 2023-11-04T05:28:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dcd2c0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc8070ce0 con 0x7f2dc80b3820 2023-11-04T05:28:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dc67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2dc80d0d70 0x7f2dc80b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57220/0 (socket says 172.21.15.3:57220) 2023-11-04T05:28:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dc67fc700 1 -- 172.21.15.3:0/4075296762 learned_addr learned my addr 172.21.15.3:0/4075296762 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.032+0000 7f2dc77fe700 1 --2- 172.21.15.3:0/4075296762 >> v2:172.21.15.188:3300/0 conn(0x7f2dc80b3820 0x7f2dc815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715056638 0 0) 0x7f2dc8070a90 con 0x7f2dc80d0d70 2023-11-04T05:28:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db00035b0 con 0x7f2dc80d0d70 2023-11-04T05:28:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144851557 0 0) 0x7f2dc80718f0 con 0x7f2dc815ba10 2023-11-04T05:28:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc8070a90 con 0x7f2dc815ba10 2023-11-04T05:28:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981291654 0 0) 0x7f2db00035b0 con 0x7f2dc80d0d70 2023-11-04T05:28:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc77fe700 1 -- 172.21.15.3:0/4075296762 >> v1:172.21.15.188:6789/0 conn(0x7f2dc815ba10 legacy=0x7f2dc80c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc77fe700 1 -- 172.21.15.3:0/4075296762 >> v1:172.21.15.3:6789/0 conn(0x7f2dc80d0d70 legacy=0x7f2dc80b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc77fe700 1 -- 172.21.15.3:0/4075296762 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc8070fc0 con 0x7f2dc80b3820 2023-11-04T05:28:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc77fe700 1 --2- 172.21.15.3:0/4075296762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80b3820 0x7f2dc815f850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009d90 tx=0x7f2dc000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88bd5592e9874d4c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000e070 con 0x7f2dc80b3820 2023-11-04T05:28:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc0003a20 con 0x7f2dc80b3820 2023-11-04T05:28:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.033+0000 7f2dc5ffb700 1 -- 172.21.15.3:0/4075296762 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc001670 con 0x7f2dc80b3820 2023-11-04T05:28:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/4075296762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80b3820 msgr2=0x7f2dc815f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/4075296762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80b3820 0x7f2dc815f850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009d90 tx=0x7f2dc000b330 comp rx=0 tx=0).stop 2023-11-04T05:28:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/4075296762 shutdown_connections 2023-11-04T05:28:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/4075296762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80b3820 0x7f2dc815f850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/4075296762 >> 172.21.15.3:0/4075296762 conn(0x7f2dc80ce110 msgr2=0x7f2dc80ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/4075296762 shutdown_connections 2023-11-04T05:28:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.034+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/4075296762 wait complete. 2023-11-04T05:28:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.035+0000 7f2dcd2c0700 1 Processor -- start 2023-11-04T05:28:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.035+0000 7f2dcd2c0700 1 -- start start 2023-11-04T05:28:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dcd2c0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dc80b3820 0x7f2dc80b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dcd2c0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80d0d70 0x7f2dc81f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dcd2c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 0x7f2dc81f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dcd2c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc81f10a0 con 0x7f2dc815ba10 2023-11-04T05:28:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dcd2c0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc81445d0 con 0x7f2dc80d0d70 2023-11-04T05:28:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dcd2c0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80b4890 con 0x7f2dc80b3820 2023-11-04T05:28:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 0x7f2dc81f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 0x7f2dc81f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51880/0 (socket says 172.21.15.3:51880) 2023-11-04T05:28:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 -- 172.21.15.3:0/1196805128 learned_addr learned my addr 172.21.15.3:0/1196805128 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc67fc700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80d0d70 0x7f2dc81f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:44.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc6ffd700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dc80b3820 0x7f2dc80b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:44.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 -- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dc80b3820 msgr2=0x7f2dc80b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dc80b3820 0x7f2dc80b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 -- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80d0d70 msgr2=0x7f2dc81f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80d0d70 0x7f2dc81f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc77fe700 1 -- 172.21.15.3:0/1196805128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc0009a70 con 0x7f2dc815ba10 2023-11-04T05:28:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc67fc700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80d0d70 0x7f2dc81f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.036+0000 7f2dc6ffd700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dc80b3820 0x7f2dc80b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.037+0000 7f2dc77fe700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 0x7f2dc81f0b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b370 tx=0x7f2dc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.037+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004600 con 0x7f2dc815ba10 2023-11-04T05:28:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.037+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc0004760 con 0x7f2dc815ba10 2023-11-04T05:28:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.037+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc80ca350 con 0x7f2dc815ba10 2023-11-04T05:28:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.037+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0012430 con 0x7f2dc815ba10 2023-11-04T05:28:44.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.037+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc8144b50 con 0x7f2dc815ba10 2023-11-04T05:28:44.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.038+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc8144d40 con 0x7f2dc815ba10 2023-11-04T05:28:44.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.038+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2dc0012590 con 0x7f2dc815ba10 2023-11-04T05:28:44.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.038+0000 7f2daf7fe700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2db406af00 0x7f2db406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:44.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.039+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2dc008ada0 con 0x7f2dc815ba10 2023-11-04T05:28:44.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.039+0000 7f2dc6ffd700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2db406af00 0x7f2db406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:44.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.039+0000 7f2dc6ffd700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2db406af00 0x7f2db406d3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2db80011e0 tx=0x7f2db8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:44.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.043+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2dc0059680 con 0x7f2dc815ba10 2023-11-04T05:28:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.288+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2dc80cee70 con 0x7f2dc815ba10 2023-11-04T05:28:44.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.889+0000 7f2daf7fe700 1 -- 172.21.15.3:0/1196805128 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2dc0016020 con 0x7f2dc815ba10 2023-11-04T05:28:44.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2db406af00 msgr2=0x7f2db406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2db406af00 0x7f2db406d3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2db80011e0 tx=0x7f2db8009040 comp rx=0 tx=0).stop 2023-11-04T05:28:44.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 msgr2=0x7f2dc81f0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:44.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 0x7f2dc81f0b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b370 tx=0x7f2dc0004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 shutdown_connections 2023-11-04T05:28:44.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2db406af00 0x7f2db406d3c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dc815ba10 0x7f2dc81f0b60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dc80b3820 0x7f2dc80b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 --2- 172.21.15.3:0/1196805128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dc80d0d70 0x7f2dc81f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:44.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 >> 172.21.15.3:0/1196805128 conn(0x7f2dc80ce110 msgr2=0x7f2dc80d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:44.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 shutdown_connections 2023-11-04T05:28:44.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:44.891+0000 7f2dcd2c0700 1 -- 172.21.15.3:0/1196805128 wait complete. 2023-11-04T05:28:44.908 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-04T05:28:44.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:45.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.035+0000 7f2072eae700 1 Processor -- start 2023-11-04T05:28:45.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.035+0000 7f2072eae700 1 -- start start 2023-11-04T05:28:45.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2072eae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c154550 0x7f206c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2072eae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f206c06d790 0x7f206c06dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2072eae700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c154930 con 0x7f206c15dc40 2023-11-04T05:28:45.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2072eae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c06e150 con 0x7f206c06d790 2023-11-04T05:28:45.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2072eae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c15e9b0 con 0x7f206c154550 2023-11-04T05:28:45.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f206c06d790 0x7f206c06dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f206c06d790 0x7f206c06dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51886/0 (socket says 172.21.15.3:51886) 2023-11-04T05:28:45.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 -- 172.21.15.3:0/3624728774 learned_addr learned my addr 172.21.15.3:0/3624728774 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:45.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2070c4a700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c154550 0x7f206c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 -- 172.21.15.3:0/3624728774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c154550 msgr2=0x7f206c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:45.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c154550 0x7f206c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 -- 172.21.15.3:0/3624728774 >> v1:172.21.15.188:6789/0 conn(0x7f206c15dc40 legacy=0x7f206c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f206bfff700 1 -- 172.21.15.3:0/3624728774 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0cddf0 con 0x7f206c06d790 2023-11-04T05:28:45.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.036+0000 7f2070c4a700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c154550 0x7f206c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:45.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f206bfff700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c06d790 0x7f206c06dc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2058006fd0 tx=0x7f2058009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de447e9f4c9bad16 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f206b7fe700 1 -- 172.21.15.3:0/3624728774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058005e30 con 0x7f206c06d790 2023-11-04T05:28:45.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f206b7fe700 1 -- 172.21.15.3:0/3624728774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2058004030 con 0x7f206c06d790 2023-11-04T05:28:45.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f206b7fe700 1 -- 172.21.15.3:0/3624728774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580043e0 con 0x7f206c06d790 2023-11-04T05:28:45.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 -- 172.21.15.3:0/3624728774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c06d790 msgr2=0x7f206c06dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c06d790 0x7f206c06dc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2058006fd0 tx=0x7f2058009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 -- 172.21.15.3:0/3624728774 shutdown_connections 2023-11-04T05:28:45.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c154550 0x7f206c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 --2- 172.21.15.3:0/3624728774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c06d790 0x7f206c06dc10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 -- 172.21.15.3:0/3624728774 >> 172.21.15.3:0/3624728774 conn(0x7f206c0cfd50 msgr2=0x7f206c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 -- 172.21.15.3:0/3624728774 shutdown_connections 2023-11-04T05:28:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.037+0000 7f2072eae700 1 -- 172.21.15.3:0/3624728774 wait complete. 2023-11-04T05:28:45.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.039+0000 7f2072eae700 1 Processor -- start 2023-11-04T05:28:45.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.039+0000 7f2072eae700 1 -- start start 2023-11-04T05:28:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.039+0000 7f2072eae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206c06d790 0x7f206c0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.039+0000 7f2072eae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 0x7f206c14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.039+0000 7f2072eae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c15dc40 0x7f206c158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f2072eae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0c7930 con 0x7f206c154550 2023-11-04T05:28:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f2072eae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0c4900 con 0x7f206c06d790 2023-11-04T05:28:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f2072eae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0c3d80 con 0x7f206c15dc40 2023-11-04T05:28:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 0x7f206c14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 0x7f206c14cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51892/0 (socket says 172.21.15.3:51892) 2023-11-04T05:28:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 -- 172.21.15.3:0/1270933559 learned_addr learned my addr 172.21.15.3:0/1270933559 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f2070c4a700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206c06d790 0x7f206c0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f207144b700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c15dc40 0x7f206c158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 -- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c15dc40 msgr2=0x7f206c158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c15dc40 0x7f206c158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 -- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206c06d790 msgr2=0x7f206c0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206c06d790 0x7f206c0c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 -- 172.21.15.3:0/1270933559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058009500 con 0x7f206c154550 2023-11-04T05:28:45.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f207144b700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c15dc40 0x7f206c158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:45.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f206bfff700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 0x7f206c14cff0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2058009130 tx=0x7f2058004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058003d60 con 0x7f206c154550 2023-11-04T05:28:45.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2058011420 con 0x7f206c154550 2023-11-04T05:28:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c0c4060 con 0x7f206c154550 2023-11-04T05:28:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580117d0 con 0x7f206c154550 2023-11-04T05:28:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.040+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c0c4620 con 0x7f206c154550 2023-11-04T05:28:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.041+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c0c7a70 con 0x7f206c154550 2023-11-04T05:28:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.042+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2058011930 con 0x7f206c154550 2023-11-04T05:28:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.042+0000 7f20697fa700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f205006af00 0x7f205006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.042+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f205808aac0 con 0x7f206c154550 2023-11-04T05:28:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.042+0000 7f2070c4a700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f205006af00 0x7f205006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.043+0000 7f2070c4a700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f205006af00 0x7f205006d3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f206000bc60 tx=0x7f2060008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.047+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2058055890 con 0x7f206c154550 2023-11-04T05:28:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.295+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f206c074df0 con 0x7f206c154550 2023-11-04T05:28:45.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.295+0000 7f20697fa700 1 -- 172.21.15.3:0/1270933559 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f2058058eb0 con 0x7f206c154550 2023-11-04T05:28:45.298 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:45.298 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:28:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.297+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f205006af00 msgr2=0x7f205006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.297+0000 7f2072eae700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f205006af00 0x7f205006d3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f206000bc60 tx=0x7f2060008040 comp rx=0 tx=0).stop 2023-11-04T05:28:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.297+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 msgr2=0x7f206c14cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.297+0000 7f2072eae700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 0x7f206c14cff0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2058009130 tx=0x7f2058004750 comp rx=0 tx=0).stop 2023-11-04T05:28:45.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 shutdown_connections 2023-11-04T05:28:45.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f205006af00 0x7f205006d3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206c06d790 0x7f206c0c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f206c15dc40 0x7f206c158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 --2- 172.21.15.3:0/1270933559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f206c154550 0x7f206c14cff0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 >> 172.21.15.3:0/1270933559 conn(0x7f206c0cfd50 msgr2=0x7f206c14e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 shutdown_connections 2023-11-04T05:28:45.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.298+0000 7f2072eae700 1 -- 172.21.15.3:0/1270933559 wait complete. 2023-11-04T05:28:45.303 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:28:45.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:45.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.434+0000 7f25df780700 1 Processor -- start 2023-11-04T05:28:45.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.434+0000 7f25df780700 1 -- start start 2023-11-04T05:28:45.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25df780700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25d806ab20 0x7f25d80728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25df780700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d80cc7a0 0x7f25d80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25df780700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d8072df0 con 0x7f25d806af00 2023-11-04T05:28:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25df780700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d80cf050 con 0x7f25d806ab20 2023-11-04T05:28:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25df780700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d80cd780 con 0x7f25d80cc7a0 2023-11-04T05:28:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25dd51c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25d806ab20 0x7f25d80728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25ddd1d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f25d806af00 0x7f25d815ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57250/0 (socket says 172.21.15.3:57250) 2023-11-04T05:28:45.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25ddd1d700 1 -- 172.21.15.3:0/2250486962 learned_addr learned my addr 172.21.15.3:0/2250486962 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:45.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25dcd1b700 1 --2- 172.21.15.3:0/2250486962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d80cc7a0 0x7f25d80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25dd51c700 1 -- 172.21.15.3:0/2250486962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d80cc7a0 msgr2=0x7f25d80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:45.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25dd51c700 1 --2- 172.21.15.3:0/2250486962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d80cc7a0 0x7f25d80cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25dd51c700 1 -- 172.21.15.3:0/2250486962 >> v1:172.21.15.3:6789/0 conn(0x7f25d806af00 legacy=0x7f25d815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25dd51c700 1 -- 172.21.15.3:0/2250486962 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80cda00 con 0x7f25d806ab20 2023-11-04T05:28:45.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.435+0000 7f25cffff700 1 -- 172.21.15.3:0/2250486962 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122150678 0 0) 0x7f25d8072df0 con 0x7f25d806af00 2023-11-04T05:28:45.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25dd51c700 1 --2- 172.21.15.3:0/2250486962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80728b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009840 tx=0x7f25c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ac76f064ba8ca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25cffff700 1 -- 172.21.15.3:0/2250486962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8011830 con 0x7f25d806ab20 2023-11-04T05:28:45.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25cffff700 1 -- 172.21.15.3:0/2250486962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c8011990 con 0x7f25d806ab20 2023-11-04T05:28:45.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25cffff700 1 -- 172.21.15.3:0/2250486962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8011d20 con 0x7f25d806ab20 2023-11-04T05:28:45.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 -- 172.21.15.3:0/2250486962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 msgr2=0x7f25d80728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 --2- 172.21.15.3:0/2250486962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80728b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009840 tx=0x7f25c8009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:45.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 -- 172.21.15.3:0/2250486962 shutdown_connections 2023-11-04T05:28:45.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 --2- 172.21.15.3:0/2250486962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80728b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 --2- 172.21.15.3:0/2250486962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d80cc7a0 0x7f25d80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 -- 172.21.15.3:0/2250486962 >> 172.21.15.3:0/2250486962 conn(0x7f25d806df90 msgr2=0x7f25d806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 -- 172.21.15.3:0/2250486962 shutdown_connections 2023-11-04T05:28:45.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.436+0000 7f25df780700 1 -- 172.21.15.3:0/2250486962 wait complete. 2023-11-04T05:28:45.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 Processor -- start 2023-11-04T05:28:45.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 -- start start 2023-11-04T05:28:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d806af00 0x7f25d8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d80cc7a0 0x7f25d81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8154ff0 con 0x7f25d806ab20 2023-11-04T05:28:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d815b340 con 0x7f25d80cc7a0 2023-11-04T05:28:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25df780700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d80c66d0 con 0x7f25d806af00 2023-11-04T05:28:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.438+0000 7f25dd51c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51904/0 (socket says 172.21.15.3:51904) 2023-11-04T05:28:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 -- 172.21.15.3:0/1564429485 learned_addr learned my addr 172.21.15.3:0/1564429485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dcd1b700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d806af00 0x7f25d8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25ddd1d700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d80cc7a0 0x7f25d81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 -- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d806af00 msgr2=0x7f25d8154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d806af00 0x7f25d8154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 -- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d80cc7a0 msgr2=0x7f25d81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d80cc7a0 0x7f25d81553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 -- 172.21.15.3:0/1564429485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80094a0 con 0x7f25d806ab20 2023-11-04T05:28:45.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dcd1b700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d806af00 0x7f25d8154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:45.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25dd51c700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80c5d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f25c80090b0 tx=0x7f25c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c80038a0 con 0x7f25d806ab20 2023-11-04T05:28:45.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c801e070 con 0x7f25d806ab20 2023-11-04T05:28:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8003c20 con 0x7f25d806ab20 2023-11-04T05:28:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d81561b0 con 0x7f25d806ab20 2023-11-04T05:28:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.439+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d815acf0 con 0x7f25d806ab20 2023-11-04T05:28:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.440+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d8074df0 con 0x7f25d806ab20 2023-11-04T05:28:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.441+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25c801a420 con 0x7f25d806ab20 2023-11-04T05:28:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.441+0000 7f25cdffb700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25bc06af30 0x7f25bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.441+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25c808a920 con 0x7f25d806ab20 2023-11-04T05:28:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.441+0000 7f25dcd1b700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25bc06af30 0x7f25bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.442+0000 7f25dcd1b700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25bc06af30 0x7f25bc06d3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25d815c7e0 tx=0x7f25d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.446+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f25c8055770 con 0x7f25d806ab20 2023-11-04T05:28:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.649+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25d80cd020 con 0x7f25d806ab20 2023-11-04T05:28:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.650+0000 7f25cdffb700 1 -- 172.21.15.3:0/1564429485 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f25c8020090 con 0x7f25d806ab20 2023-11-04T05:28:45.653 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:45.653 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25bc06af30 msgr2=0x7f25bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25bc06af30 0x7f25bc06d3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25d815c7e0 tx=0x7f25d4008040 comp rx=0 tx=0).stop 2023-11-04T05:28:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 msgr2=0x7f25d80c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80c5d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f25c80090b0 tx=0x7f25c8004750 comp rx=0 tx=0).stop 2023-11-04T05:28:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 shutdown_connections 2023-11-04T05:28:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25bc06af30 0x7f25bc06d3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d80cc7a0 0x7f25d81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25d806ab20 0x7f25d80c5d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 --2- 172.21.15.3:0/1564429485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d806af00 0x7f25d8154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.652+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 >> 172.21.15.3:0/1564429485 conn(0x7f25d806df90 msgr2=0x7f25d80c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.653+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 shutdown_connections 2023-11-04T05:28:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.653+0000 7f25df780700 1 -- 172.21.15.3:0/1564429485 wait complete. 2023-11-04T05:28:45.667 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-04T05:28:45.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-04T05:28:45.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.790+0000 7fa14369f700 1 Processor -- start 2023-11-04T05:28:45.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.790+0000 7fa14369f700 1 -- start start 2023-11-04T05:28:45.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa14369f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa13c0cc7a0 0x7fa13c0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa14369f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06af00 0x7fa13c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa14369f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c072df0 con 0x7fa13c06ab20 2023-11-04T05:28:45.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa14369f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0cf050 con 0x7fa13c0cc7a0 2023-11-04T05:28:45.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa14369f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0cda00 con 0x7fa13c06af00 2023-11-04T05:28:45.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa140c3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa13c0cc7a0 0x7fa13c0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa140c3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa13c0cc7a0 0x7fa13c0728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51916/0 (socket says 172.21.15.3:51916) 2023-11-04T05:28:45.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.791+0000 7fa140c3a700 1 -- 172.21.15.3:0/2573439391 learned_addr learned my addr 172.21.15.3:0/2573439391 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:45.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa141c3c700 1 --2- 172.21.15.3:0/2573439391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06af00 0x7fa13c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa140c3a700 1 -- 172.21.15.3:0/2573439391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06af00 msgr2=0x7fa13c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:45.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa140c3a700 1 --2- 172.21.15.3:0/2573439391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06af00 0x7fa13c0d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa140c3a700 1 -- 172.21.15.3:0/2573439391 >> v1:172.21.15.188:6789/0 conn(0x7fa13c06ab20 legacy=0x7fa13c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:45.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa140c3a700 1 -- 172.21.15.3:0/2573439391 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0cdce0 con 0x7fa13c0cc7a0 2023-11-04T05:28:45.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa140c3a700 1 --2- 172.21.15.3:0/2573439391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0728b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1380098e0 tx=0x7fa138009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ad09c19a064320 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa133fff700 1 -- 172.21.15.3:0/2573439391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800b8a0 con 0x7fa13c0cc7a0 2023-11-04T05:28:45.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa133fff700 1 -- 172.21.15.3:0/2573439391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800ba00 con 0x7fa13c0cc7a0 2023-11-04T05:28:45.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.792+0000 7fa133fff700 1 -- 172.21.15.3:0/2573439391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800bdb0 con 0x7fa13c0cc7a0 2023-11-04T05:28:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 -- 172.21.15.3:0/2573439391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0cc7a0 msgr2=0x7fa13c0728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 --2- 172.21.15.3:0/2573439391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0728b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1380098e0 tx=0x7fa138009bf0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 -- 172.21.15.3:0/2573439391 shutdown_connections 2023-11-04T05:28:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 --2- 172.21.15.3:0/2573439391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06af00 0x7fa13c0d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 --2- 172.21.15.3:0/2573439391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0728b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 -- 172.21.15.3:0/2573439391 >> 172.21.15.3:0/2573439391 conn(0x7fa13c06df90 msgr2=0x7fa13c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:45.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 -- 172.21.15.3:0/2573439391 shutdown_connections 2023-11-04T05:28:45.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.793+0000 7fa14369f700 1 -- 172.21.15.3:0/2573439391 wait complete. 2023-11-04T05:28:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.794+0000 7fa14369f700 1 Processor -- start 2023-11-04T05:28:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.794+0000 7fa14369f700 1 -- start start 2023-11-04T05:28:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14369f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06ab20 0x7fa13c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14369f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 0x7fa13c0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14369f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14369f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0af200 con 0x7fa13c06af00 2023-11-04T05:28:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14369f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0ae8f0 con 0x7fa13c0cc7a0 2023-11-04T05:28:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14369f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c159f70 con 0x7fa13c06ab20 2023-11-04T05:28:45.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 0x7fa13c0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 0x7fa13c0c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51920/0 (socket says 172.21.15.3:51920) 2023-11-04T05:28:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 -- 172.21.15.3:0/1925665419 learned_addr learned my addr 172.21.15.3:0/1925665419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa141c3c700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa14143b700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06ab20 0x7fa13c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 -- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06ab20 msgr2=0x7fa13c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06ab20 0x7fa13c15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 -- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c0cc7a0 msgr2=0x7fa13c0c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 -- 172.21.15.3:0/1925665419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380094f0 con 0x7fa13c06af00 2023-11-04T05:28:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa141c3c700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa140c3a700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 0x7fa13c0c6170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa138009c30 tx=0x7fa138004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.795+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138004440 con 0x7fa13c06af00 2023-11-04T05:28:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.796+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1380118c0 con 0x7fa13c06af00 2023-11-04T05:28:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.796+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0c6cc0 con 0x7fa13c06af00 2023-11-04T05:28:45.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.796+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138011c70 con 0x7fa13c06af00 2023-11-04T05:28:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.796+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c0aee20 con 0x7fa13c06af00 2023-11-04T05:28:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.797+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c0af340 con 0x7fa13c06af00 2023-11-04T05:28:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.797+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1380036a0 con 0x7fa13c06af00 2023-11-04T05:28:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.797+0000 7fa131ffb700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa12006af00 0x7fa12006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:45.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.798+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa138021030 con 0x7fa13c06af00 2023-11-04T05:28:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.798+0000 7fa14143b700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa12006af00 0x7fa12006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.798+0000 7fa14143b700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa12006af00 0x7fa12006d3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa12c000f60 tx=0x7fa12c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:45.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:45.802+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa13c0af340 con 0x7fa13c06af00 2023-11-04T05:28:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.002+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 --> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa13c072390 con 0x7fa12006af00 2023-11-04T05:28:46.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.013+0000 7fa131ffb700 1 -- 172.21.15.3:0/1925665419 <== mgr.4099 v2:172.21.15.188:6832/3102759174 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345262 (secure 0 0 0) 0x7fa12c001490 con 0x7fa12006af00 2023-11-04T05:28:46.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa12006af00 msgr2=0x7fa12006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa12006af00 0x7fa12006d3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa12c000f60 tx=0x7fa12c009040 comp rx=0 tx=0).stop 2023-11-04T05:28:46.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 msgr2=0x7fa13c0c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 0x7fa13c0c6170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa138009c30 tx=0x7fa138004750 comp rx=0 tx=0).stop 2023-11-04T05:28:46.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 shutdown_connections 2023-11-04T05:28:46.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa12006af00 0x7fa12006d3c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c0cc7a0 0x7fa13c0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c06ab20 0x7fa13c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 --2- 172.21.15.3:0/1925665419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c06af00 0x7fa13c0c6170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 >> 172.21.15.3:0/1925665419 conn(0x7fa13c06df90 msgr2=0x7fa13c0cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:46.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.017+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 shutdown_connections 2023-11-04T05:28:46.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.018+0000 7fa14369f700 1 -- 172.21.15.3:0/1925665419 wait complete. 2023-11-04T05:28:46.026 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-11-04T05:28:46.031 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-11-04T05:28:45.756017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2045232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003796"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835820+0000","last_change":"2023-11-04T05:28:21.455216+0000","last_active":"2023-11-04T05:28:25.835820+0000","last_peered":"2023-11-04T05:28:25.835820+0000","last_clean":"2023-11-04T05:28:25.835820+0000","last_became_active":"2023-11-04T05:28:21.453964+0000","last_became_peered":"2023-11-04T05:28:21.453964+0000","last_unstale":"2023-11-04T05:28:25.835820+0000","last_undegraded":"2023-11-04T05:28:25.835820+0000","last_fullsized":"2023-11-04T05:28:25.835820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858340+0000","last_change":"2023-11-04T05:28:21.458543+0000","last_active":"2023-11-04T05:28:25.858340+0000","last_peered":"2023-11-04T05:28:25.858340+0000","last_clean":"2023-11-04T05:28:25.858340+0000","last_became_active":"2023-11-04T05:28:21.458174+0000","last_became_peered":"2023-11-04T05:28:21.458174+0000","last_unstale":"2023-11-04T05:28:25.858340+0000","last_undegraded":"2023-11-04T05:28:25.858340+0000","last_fullsized":"2023-11-04T05:28:25.858340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842597+0000","last_change":"2023-11-04T05:28:21.457190+0000","last_active":"2023-11-04T05:28:25.842597+0000","last_peered":"2023-11-04T05:28:25.842597+0000","last_clean":"2023-11-04T05:28:25.842597+0000","last_became_active":"2023-11-04T05:28:21.456901+0000","last_became_peered":"2023-11-04T05:28:21.456901+0000","last_unstale":"2023-11-04T05:28:25.842597+0000","last_undegraded":"2023-11-04T05:28:25.842597+0000","last_fullsized":"2023-11-04T05:28:25.842597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:00.273535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840758+0000","last_change":"2023-11-04T05:28:21.464308+0000","last_active":"2023-11-04T05:28:25.840758+0000","last_peered":"2023-11-04T05:28:25.840758+0000","last_clean":"2023-11-04T05:28:25.840758+0000","last_became_active":"2023-11-04T05:28:21.464035+0000","last_became_peered":"2023-11-04T05:28:21.464035+0000","last_unstale":"2023-11-04T05:28:25.840758+0000","last_undegraded":"2023-11-04T05:28:25.840758+0000","last_fullsized":"2023-11-04T05:28:25.840758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:21.443272+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:21.442885+0000","last_became_peered":"2023-11-04T05:28:21.442885+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839412+0000","last_change":"2023-11-04T05:28:21.461562+0000","last_active":"2023-11-04T05:28:25.839412+0000","last_peered":"2023-11-04T05:28:25.839412+0000","last_clean":"2023-11-04T05:28:25.839412+0000","last_became_active":"2023-11-04T05:28:21.461276+0000","last_became_peered":"2023-11-04T05:28:21.461276+0000","last_unstale":"2023-11-04T05:28:25.839412+0000","last_undegraded":"2023-11-04T05:28:25.839412+0000","last_fullsized":"2023-11-04T05:28:25.839412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834303+0000","last_change":"2023-11-04T05:28:21.456621+0000","last_active":"2023-11-04T05:28:25.834303+0000","last_peered":"2023-11-04T05:28:25.834303+0000","last_clean":"2023-11-04T05:28:25.834303+0000","last_became_active":"2023-11-04T05:28:21.456356+0000","last_became_peered":"2023-11-04T05:28:21.456356+0000","last_unstale":"2023-11-04T05:28:25.834303+0000","last_undegraded":"2023-11-04T05:28:25.834303+0000","last_fullsized":"2023-11-04T05:28:25.834303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841746+0000","last_change":"2023-11-04T05:28:21.463624+0000","last_active":"2023-11-04T05:28:25.841746+0000","last_peered":"2023-11-04T05:28:25.841746+0000","last_clean":"2023-11-04T05:28:25.841746+0000","last_became_active":"2023-11-04T05:28:21.463336+0000","last_became_peered":"2023-11-04T05:28:21.463336+0000","last_unstale":"2023-11-04T05:28:25.841746+0000","last_undegraded":"2023-11-04T05:28:25.841746+0000","last_fullsized":"2023-11-04T05:28:25.841746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835730+0000","last_change":"2023-11-04T05:28:21.466992+0000","last_active":"2023-11-04T05:28:25.835730+0000","last_peered":"2023-11-04T05:28:25.835730+0000","last_clean":"2023-11-04T05:28:25.835730+0000","last_became_active":"2023-11-04T05:28:21.466638+0000","last_became_peered":"2023-11-04T05:28:21.466638+0000","last_unstale":"2023-11-04T05:28:25.835730+0000","last_undegraded":"2023-11-04T05:28:25.835730+0000","last_fullsized":"2023-11-04T05:28:25.835730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:01:55.625411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835857+0000","last_change":"2023-11-04T05:28:21.466265+0000","last_active":"2023-11-04T05:28:25.835857+0000","last_peered":"2023-11-04T05:28:25.835857+0000","last_clean":"2023-11-04T05:28:25.835857+0000","last_became_active":"2023-11-04T05:28:21.466080+0000","last_became_peered":"2023-11-04T05:28:21.466080+0000","last_unstale":"2023-11-04T05:28:25.835857+0000","last_undegraded":"2023-11-04T05:28:25.835857+0000","last_fullsized":"2023-11-04T05:28:25.835857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835259+0000","last_change":"2023-11-04T05:28:21.465839+0000","last_active":"2023-11-04T05:28:25.835259+0000","last_peered":"2023-11-04T05:28:25.835259+0000","last_clean":"2023-11-04T05:28:25.835259+0000","last_became_active":"2023-11-04T05:28:21.465571+0000","last_became_peered":"2023-11-04T05:28:21.465571+0000","last_unstale":"2023-11-04T05:28:25.835259+0000","last_undegraded":"2023-11-04T05:28:25.835259+0000","last_fullsized":"2023-11-04T05:28:25.835259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843187+0000","last_change":"2023-11-04T05:28:21.459690+0000","last_active":"2023-11-04T05:28:25.843187+0000","last_peered":"2023-11-04T05:28:25.843187+0000","last_clean":"2023-11-04T05:28:25.843187+0000","last_became_active":"2023-11-04T05:28:21.459406+0000","last_became_peered":"2023-11-04T05:28:21.459406+0000","last_unstale":"2023-11-04T05:28:25.843187+0000","last_undegraded":"2023-11-04T05:28:25.843187+0000","last_fullsized":"2023-11-04T05:28:25.843187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840245+0000","last_change":"2023-11-04T05:28:21.452009+0000","last_active":"2023-11-04T05:28:25.840245+0000","last_peered":"2023-11-04T05:28:25.840245+0000","last_clean":"2023-11-04T05:28:25.840245+0000","last_became_active":"2023-11-04T05:28:21.451701+0000","last_became_peered":"2023-11-04T05:28:21.451701+0000","last_unstale":"2023-11-04T05:28:25.840245+0000","last_undegraded":"2023-11-04T05:28:25.840245+0000","last_fullsized":"2023-11-04T05:28:25.840245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842267+0000","last_change":"2023-11-04T05:28:21.449054+0000","last_active":"2023-11-04T05:28:25.842267+0000","last_peered":"2023-11-04T05:28:25.842267+0000","last_clean":"2023-11-04T05:28:25.842267+0000","last_became_active":"2023-11-04T05:28:21.448593+0000","last_became_peered":"2023-11-04T05:28:21.448593+0000","last_unstale":"2023-11-04T05:28:25.842267+0000","last_undegraded":"2023-11-04T05:28:25.842267+0000","last_fullsized":"2023-11-04T05:28:25.842267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:16:04.673589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857682+0000","last_change":"2023-11-04T05:28:21.459859+0000","last_active":"2023-11-04T05:28:25.857682+0000","last_peered":"2023-11-04T05:28:25.857682+0000","last_clean":"2023-11-04T05:28:25.857682+0000","last_became_active":"2023-11-04T05:28:21.459569+0000","last_became_peered":"2023-11-04T05:28:21.459569+0000","last_unstale":"2023-11-04T05:28:25.857682+0000","last_undegraded":"2023-11-04T05:28:25.857682+0000","last_fullsized":"2023-11-04T05:28:25.857682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839549+0000","last_change":"2023-11-04T05:28:21.461640+0000","last_active":"2023-11-04T05:28:25.839549+0000","last_peered":"2023-11-04T05:28:25.839549+0000","last_clean":"2023-11-04T05:28:25.839549+0000","last_became_active":"2023-11-04T05:28:21.461239+0000","last_became_peered":"2023-11-04T05:28:21.461239+0000","last_unstale":"2023-11-04T05:28:25.839549+0000","last_undegraded":"2023-11-04T05:28:25.839549+0000","last_fullsized":"2023-11-04T05:28:25.839549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841199+0000","last_change":"2023-11-04T05:28:21.463999+0000","last_active":"2023-11-04T05:28:25.841199+0000","last_peered":"2023-11-04T05:28:25.841199+0000","last_clean":"2023-11-04T05:28:25.841199+0000","last_became_active":"2023-11-04T05:28:21.463743+0000","last_became_peered":"2023-11-04T05:28:21.463743+0000","last_unstale":"2023-11-04T05:28:25.841199+0000","last_undegraded":"2023-11-04T05:28:25.841199+0000","last_fullsized":"2023-11-04T05:28:25.841199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846908+0000","last_change":"2023-11-04T05:28:21.454396+0000","last_active":"2023-11-04T05:28:25.846908+0000","last_peered":"2023-11-04T05:28:25.846908+0000","last_clean":"2023-11-04T05:28:25.846908+0000","last_became_active":"2023-11-04T05:28:21.454074+0000","last_became_peered":"2023-11-04T05:28:21.454074+0000","last_unstale":"2023-11-04T05:28:25.846908+0000","last_undegraded":"2023-11-04T05:28:25.846908+0000","last_fullsized":"2023-11-04T05:28:25.846908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835087+0000","last_change":"2023-11-04T05:28:21.441379+0000","last_active":"2023-11-04T05:28:25.835087+0000","last_peered":"2023-11-04T05:28:25.835087+0000","last_clean":"2023-11-04T05:28:25.835087+0000","last_became_active":"2023-11-04T05:28:21.441074+0000","last_became_peered":"2023-11-04T05:28:21.441074+0000","last_unstale":"2023-11-04T05:28:25.835087+0000","last_undegraded":"2023-11-04T05:28:25.835087+0000","last_fullsized":"2023-11-04T05:28:25.835087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220068+0000","last_change":"2023-11-04T05:28:25.856956+0000","last_active":"2023-11-04T05:28:28.220068+0000","last_peered":"2023-11-04T05:28:28.220068+0000","last_clean":"2023-11-04T05:28:28.220068+0000","last_became_active":"2023-11-04T05:28:25.856458+0000","last_became_peered":"2023-11-04T05:28:25.856458+0000","last_unstale":"2023-11-04T05:28:28.220068+0000","last_undegraded":"2023-11-04T05:28:28.220068+0000","last_fullsized":"2023-11-04T05:28:28.220068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:01:17.491615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842941+0000","last_change":"2023-11-04T05:28:20.455408+0000","last_active":"2023-11-04T05:28:25.842941+0000","last_peered":"2023-11-04T05:28:25.842941+0000","last_clean":"2023-11-04T05:28:25.842941+0000","last_became_active":"2023-11-04T05:28:20.455106+0000","last_became_peered":"2023-11-04T05:28:20.455106+0000","last_unstale":"2023-11-04T05:28:25.842941+0000","last_undegraded":"2023-11-04T05:28:25.842941+0000","last_fullsized":"2023-11-04T05:28:25.842941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:09:15.898778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840543+0000","last_change":"2023-11-04T05:28:21.452315+0000","last_active":"2023-11-04T05:28:25.840543+0000","last_peered":"2023-11-04T05:28:25.840543+0000","last_clean":"2023-11-04T05:28:25.840543+0000","last_became_active":"2023-11-04T05:28:21.451728+0000","last_became_peered":"2023-11-04T05:28:21.451728+0000","last_unstale":"2023-11-04T05:28:25.840543+0000","last_undegraded":"2023-11-04T05:28:25.840543+0000","last_fullsized":"2023-11-04T05:28:25.840543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742885+0000","last_change":"2023-11-04T05:28:25.861875+0000","last_active":"2023-11-04T05:28:28.742885+0000","last_peered":"2023-11-04T05:28:28.742885+0000","last_clean":"2023-11-04T05:28:28.742885+0000","last_became_active":"2023-11-04T05:28:25.861458+0000","last_became_peered":"2023-11-04T05:28:25.861458+0000","last_unstale":"2023-11-04T05:28:28.742885+0000","last_undegraded":"2023-11-04T05:28:28.742885+0000","last_fullsized":"2023-11-04T05:28:28.742885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.153647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840648+0000","last_change":"2023-11-04T05:28:20.456506+0000","last_active":"2023-11-04T05:28:25.840648+0000","last_peered":"2023-11-04T05:28:25.840648+0000","last_clean":"2023-11-04T05:28:25.840648+0000","last_became_active":"2023-11-04T05:28:20.456086+0000","last_became_peered":"2023-11-04T05:28:20.456086+0000","last_unstale":"2023-11-04T05:28:25.840648+0000","last_undegraded":"2023-11-04T05:28:25.840648+0000","last_fullsized":"2023-11-04T05:28:25.840648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842845+0000","last_change":"2023-11-04T05:28:21.464433+0000","last_active":"2023-11-04T05:28:25.842845+0000","last_peered":"2023-11-04T05:28:25.842845+0000","last_clean":"2023-11-04T05:28:25.842845+0000","last_became_active":"2023-11-04T05:28:21.464133+0000","last_became_peered":"2023-11-04T05:28:21.464133+0000","last_unstale":"2023-11-04T05:28:25.842845+0000","last_undegraded":"2023-11-04T05:28:25.842845+0000","last_fullsized":"2023-11-04T05:28:25.842845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:26:07.829857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743096+0000","last_change":"2023-11-04T05:28:25.858693+0000","last_active":"2023-11-04T05:28:28.743096+0000","last_peered":"2023-11-04T05:28:28.743096+0000","last_clean":"2023-11-04T05:28:28.743096+0000","last_became_active":"2023-11-04T05:28:25.858151+0000","last_became_peered":"2023-11-04T05:28:25.858151+0000","last_unstale":"2023-11-04T05:28:28.743096+0000","last_undegraded":"2023-11-04T05:28:28.743096+0000","last_fullsized":"2023-11-04T05:28:28.743096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:33:57.728742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:20.448994+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:20.448722+0000","last_became_peered":"2023-11-04T05:28:20.448722+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839002+0000","last_change":"2023-11-04T05:28:21.464488+0000","last_active":"2023-11-04T05:28:25.839002+0000","last_peered":"2023-11-04T05:28:25.839002+0000","last_clean":"2023-11-04T05:28:25.839002+0000","last_became_active":"2023-11-04T05:28:21.462679+0000","last_became_peered":"2023-11-04T05:28:21.462679+0000","last_unstale":"2023-11-04T05:28:25.839002+0000","last_undegraded":"2023-11-04T05:28:25.839002+0000","last_fullsized":"2023-11-04T05:28:25.839002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743030+0000","last_change":"2023-11-04T05:28:25.854516+0000","last_active":"2023-11-04T05:28:28.743030+0000","last_peered":"2023-11-04T05:28:28.743030+0000","last_clean":"2023-11-04T05:28:28.743030+0000","last_became_active":"2023-11-04T05:28:25.854162+0000","last_became_peered":"2023-11-04T05:28:25.854162+0000","last_unstale":"2023-11-04T05:28:28.743030+0000","last_undegraded":"2023-11-04T05:28:28.743030+0000","last_fullsized":"2023-11-04T05:28:28.743030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:06.645428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834998+0000","last_change":"2023-11-04T05:28:20.452770+0000","last_active":"2023-11-04T05:28:25.834998+0000","last_peered":"2023-11-04T05:28:25.834998+0000","last_clean":"2023-11-04T05:28:25.834998+0000","last_became_active":"2023-11-04T05:28:20.452404+0000","last_became_peered":"2023-11-04T05:28:20.452404+0000","last_unstale":"2023-11-04T05:28:25.834998+0000","last_undegraded":"2023-11-04T05:28:25.834998+0000","last_fullsized":"2023-11-04T05:28:25.834998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842144+0000","last_change":"2023-11-04T05:28:21.463294+0000","last_active":"2023-11-04T05:28:25.842144+0000","last_peered":"2023-11-04T05:28:25.842144+0000","last_clean":"2023-11-04T05:28:25.842144+0000","last_became_active":"2023-11-04T05:28:21.463028+0000","last_became_peered":"2023-11-04T05:28:21.463028+0000","last_unstale":"2023-11-04T05:28:25.842144+0000","last_undegraded":"2023-11-04T05:28:25.842144+0000","last_fullsized":"2023-11-04T05:28:25.842144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220306+0000","last_change":"2023-11-04T05:28:25.864434+0000","last_active":"2023-11-04T05:28:28.220306+0000","last_peered":"2023-11-04T05:28:28.220306+0000","last_clean":"2023-11-04T05:28:28.220306+0000","last_became_active":"2023-11-04T05:28:25.863978+0000","last_became_peered":"2023-11-04T05:28:25.863978+0000","last_unstale":"2023-11-04T05:28:28.220306+0000","last_undegraded":"2023-11-04T05:28:28.220306+0000","last_fullsized":"2023-11-04T05:28:28.220306+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:22:02.298285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840436+0000","last_change":"2023-11-04T05:28:20.451845+0000","last_active":"2023-11-04T05:28:25.840436+0000","last_peered":"2023-11-04T05:28:25.840436+0000","last_clean":"2023-11-04T05:28:25.840436+0000","last_became_active":"2023-11-04T05:28:20.451350+0000","last_became_peered":"2023-11-04T05:28:20.451350+0000","last_unstale":"2023-11-04T05:28:25.840436+0000","last_undegraded":"2023-11-04T05:28:25.840436+0000","last_fullsized":"2023-11-04T05:28:25.840436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.837664+0000","last_change":"2023-11-04T05:28:21.454758+0000","last_active":"2023-11-04T05:28:25.837664+0000","last_peered":"2023-11-04T05:28:25.837664+0000","last_clean":"2023-11-04T05:28:25.837664+0000","last_became_active":"2023-11-04T05:28:21.449696+0000","last_became_peered":"2023-11-04T05:28:21.449696+0000","last_unstale":"2023-11-04T05:28:25.837664+0000","last_undegraded":"2023-11-04T05:28:25.837664+0000","last_fullsized":"2023-11-04T05:28:25.837664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:13:54.163937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.177060+0000","last_change":"2023-11-04T05:28:25.869885+0000","last_active":"2023-11-04T05:28:28.177060+0000","last_peered":"2023-11-04T05:28:28.177060+0000","last_clean":"2023-11-04T05:28:28.177060+0000","last_became_active":"2023-11-04T05:28:25.869295+0000","last_became_peered":"2023-11-04T05:28:25.869295+0000","last_unstale":"2023-11-04T05:28:28.177060+0000","last_undegraded":"2023-11-04T05:28:28.177060+0000","last_fullsized":"2023-11-04T05:28:28.177060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:02:41.884046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834648+0000","last_change":"2023-11-04T05:28:20.445804+0000","last_active":"2023-11-04T05:28:25.834648+0000","last_peered":"2023-11-04T05:28:25.834648+0000","last_clean":"2023-11-04T05:28:25.834648+0000","last_became_active":"2023-11-04T05:28:20.445476+0000","last_became_peered":"2023-11-04T05:28:20.445476+0000","last_unstale":"2023-11-04T05:28:25.834648+0000","last_undegraded":"2023-11-04T05:28:25.834648+0000","last_fullsized":"2023-11-04T05:28:25.834648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:22:03.756669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842664+0000","last_change":"2023-11-04T05:28:21.460996+0000","last_active":"2023-11-04T05:28:25.842664+0000","last_peered":"2023-11-04T05:28:25.842664+0000","last_clean":"2023-11-04T05:28:25.842664+0000","last_became_active":"2023-11-04T05:28:21.460451+0000","last_became_peered":"2023-11-04T05:28:21.460451+0000","last_unstale":"2023-11-04T05:28:25.842664+0000","last_undegraded":"2023-11-04T05:28:25.842664+0000","last_fullsized":"2023-11-04T05:28:25.842664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:02.249647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838965+0000","last_change":"2023-11-04T05:28:25.863990+0000","last_active":"2023-11-04T05:28:26.838965+0000","last_peered":"2023-11-04T05:28:26.838965+0000","last_clean":"2023-11-04T05:28:26.838965+0000","last_became_active":"2023-11-04T05:28:25.863587+0000","last_became_peered":"2023-11-04T05:28:25.863587+0000","last_unstale":"2023-11-04T05:28:26.838965+0000","last_undegraded":"2023-11-04T05:28:26.838965+0000","last_fullsized":"2023-11-04T05:28:26.838965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841913+0000","last_change":"2023-11-04T05:28:20.448359+0000","last_active":"2023-11-04T05:28:25.841913+0000","last_peered":"2023-11-04T05:28:25.841913+0000","last_clean":"2023-11-04T05:28:25.841913+0000","last_became_active":"2023-11-04T05:28:20.448037+0000","last_became_peered":"2023-11-04T05:28:20.448037+0000","last_unstale":"2023-11-04T05:28:25.841913+0000","last_undegraded":"2023-11-04T05:28:25.841913+0000","last_fullsized":"2023-11-04T05:28:25.841913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:02:08.721929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839909+0000","last_change":"2023-11-04T05:28:21.459362+0000","last_active":"2023-11-04T05:28:25.839909+0000","last_peered":"2023-11-04T05:28:25.839909+0000","last_clean":"2023-11-04T05:28:25.839909+0000","last_became_active":"2023-11-04T05:28:21.458878+0000","last_became_peered":"2023-11-04T05:28:21.458878+0000","last_unstale":"2023-11-04T05:28:25.839909+0000","last_undegraded":"2023-11-04T05:28:25.839909+0000","last_fullsized":"2023-11-04T05:28:25.839909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919977+0000","last_change":"2023-11-04T05:28:25.865090+0000","last_active":"2023-11-04T05:28:27.919977+0000","last_peered":"2023-11-04T05:28:27.919977+0000","last_clean":"2023-11-04T05:28:27.919977+0000","last_became_active":"2023-11-04T05:28:25.864492+0000","last_became_peered":"2023-11-04T05:28:25.864492+0000","last_unstale":"2023-11-04T05:28:27.919977+0000","last_undegraded":"2023-11-04T05:28:27.919977+0000","last_fullsized":"2023-11-04T05:28:27.919977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:52:12.074596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839654+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.839654+0000","last_peered":"2023-11-04T05:28:25.839654+0000","last_clean":"2023-11-04T05:28:25.839654+0000","last_became_active":"2023-11-04T05:28:20.451322+0000","last_became_peered":"2023-11-04T05:28:20.451322+0000","last_unstale":"2023-11-04T05:28:25.839654+0000","last_undegraded":"2023-11-04T05:28:25.839654+0000","last_fullsized":"2023-11-04T05:28:25.839654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:16.770342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857938+0000","last_change":"2023-11-04T05:28:21.446085+0000","last_active":"2023-11-04T05:28:25.857938+0000","last_peered":"2023-11-04T05:28:25.857938+0000","last_clean":"2023-11-04T05:28:25.857938+0000","last_became_active":"2023-11-04T05:28:21.445831+0000","last_became_peered":"2023-11-04T05:28:21.445831+0000","last_unstale":"2023-11-04T05:28:25.857938+0000","last_undegraded":"2023-11-04T05:28:25.857938+0000","last_fullsized":"2023-11-04T05:28:25.857938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219926+0000","last_change":"2023-11-04T05:28:25.853036+0000","last_active":"2023-11-04T05:28:28.219926+0000","last_peered":"2023-11-04T05:28:28.219926+0000","last_clean":"2023-11-04T05:28:28.219926+0000","last_became_active":"2023-11-04T05:28:25.852217+0000","last_became_peered":"2023-11-04T05:28:25.852217+0000","last_unstale":"2023-11-04T05:28:28.219926+0000","last_undegraded":"2023-11-04T05:28:28.219926+0000","last_fullsized":"2023-11-04T05:28:28.219926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835077+0000","last_change":"2023-11-04T05:28:20.448588+0000","last_active":"2023-11-04T05:28:25.835077+0000","last_peered":"2023-11-04T05:28:25.835077+0000","last_clean":"2023-11-04T05:28:25.835077+0000","last_became_active":"2023-11-04T05:28:20.448330+0000","last_became_peered":"2023-11-04T05:28:20.448330+0000","last_unstale":"2023-11-04T05:28:25.835077+0000","last_undegraded":"2023-11-04T05:28:25.835077+0000","last_fullsized":"2023-11-04T05:28:25.835077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841018+0000","last_change":"2023-11-04T05:28:21.462557+0000","last_active":"2023-11-04T05:28:25.841018+0000","last_peered":"2023-11-04T05:28:25.841018+0000","last_clean":"2023-11-04T05:28:25.841018+0000","last_became_active":"2023-11-04T05:28:21.462247+0000","last_became_peered":"2023-11-04T05:28:21.462247+0000","last_unstale":"2023-11-04T05:28:25.841018+0000","last_undegraded":"2023-11-04T05:28:25.841018+0000","last_fullsized":"2023-11-04T05:28:25.841018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219973+0000","last_change":"2023-11-04T05:28:25.860094+0000","last_active":"2023-11-04T05:28:28.219973+0000","last_peered":"2023-11-04T05:28:28.219973+0000","last_clean":"2023-11-04T05:28:28.219973+0000","last_became_active":"2023-11-04T05:28:25.859164+0000","last_became_peered":"2023-11-04T05:28:25.859164+0000","last_unstale":"2023-11-04T05:28:28.219973+0000","last_undegraded":"2023-11-04T05:28:28.219973+0000","last_fullsized":"2023-11-04T05:28:28.219973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840932+0000","last_change":"2023-11-04T05:28:20.451529+0000","last_active":"2023-11-04T05:28:25.840932+0000","last_peered":"2023-11-04T05:28:25.840932+0000","last_clean":"2023-11-04T05:28:25.840932+0000","last_became_active":"2023-11-04T05:28:20.451236+0000","last_became_peered":"2023-11-04T05:28:20.451236+0000","last_unstale":"2023-11-04T05:28:25.840932+0000","last_undegraded":"2023-11-04T05:28:25.840932+0000","last_fullsized":"2023-11-04T05:28:25.840932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843021+0000","last_change":"2023-11-04T05:28:21.464721+0000","last_active":"2023-11-04T05:28:25.843021+0000","last_peered":"2023-11-04T05:28:25.843021+0000","last_clean":"2023-11-04T05:28:25.843021+0000","last_became_active":"2023-11-04T05:28:21.464403+0000","last_became_peered":"2023-11-04T05:28:21.464403+0000","last_unstale":"2023-11-04T05:28:25.843021+0000","last_undegraded":"2023-11-04T05:28:25.843021+0000","last_fullsized":"2023-11-04T05:28:25.843021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:40.757866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743335+0000","last_change":"2023-11-04T05:28:25.858588+0000","last_active":"2023-11-04T05:28:28.743335+0000","last_peered":"2023-11-04T05:28:28.743335+0000","last_clean":"2023-11-04T05:28:28.743335+0000","last_became_active":"2023-11-04T05:28:25.858202+0000","last_became_peered":"2023-11-04T05:28:25.858202+0000","last_unstale":"2023-11-04T05:28:28.743335+0000","last_undegraded":"2023-11-04T05:28:28.743335+0000","last_fullsized":"2023-11-04T05:28:28.743335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:36:16.890049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858629+0000","last_change":"2023-11-04T05:28:20.451067+0000","last_active":"2023-11-04T05:28:25.858629+0000","last_peered":"2023-11-04T05:28:25.858629+0000","last_clean":"2023-11-04T05:28:25.858629+0000","last_became_active":"2023-11-04T05:28:20.450732+0000","last_became_peered":"2023-11-04T05:28:20.450732+0000","last_unstale":"2023-11-04T05:28:25.858629+0000","last_undegraded":"2023-11-04T05:28:25.858629+0000","last_fullsized":"2023-11-04T05:28:25.858629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843093+0000","last_change":"2023-11-04T05:28:21.457579+0000","last_active":"2023-11-04T05:28:25.843093+0000","last_peered":"2023-11-04T05:28:25.843093+0000","last_clean":"2023-11-04T05:28:25.843093+0000","last_became_active":"2023-11-04T05:28:21.457314+0000","last_became_peered":"2023-11-04T05:28:21.457314+0000","last_unstale":"2023-11-04T05:28:25.843093+0000","last_undegraded":"2023-11-04T05:28:25.843093+0000","last_fullsized":"2023-11-04T05:28:25.843093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832422+0000","last_change":"2023-11-04T05:28:25.863159+0000","last_active":"2023-11-04T05:28:28.832422+0000","last_peered":"2023-11-04T05:28:28.832422+0000","last_clean":"2023-11-04T05:28:28.832422+0000","last_became_active":"2023-11-04T05:28:25.862645+0000","last_became_peered":"2023-11-04T05:28:25.862645+0000","last_unstale":"2023-11-04T05:28:28.832422+0000","last_undegraded":"2023-11-04T05:28:28.832422+0000","last_fullsized":"2023-11-04T05:28:28.832422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:41.867932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:31.743983+0000","last_change":"2023-11-04T05:28:20.449394+0000","last_active":"2023-11-04T05:28:31.743983+0000","last_peered":"2023-11-04T05:28:31.743983+0000","last_clean":"2023-11-04T05:28:31.743983+0000","last_became_active":"2023-11-04T05:28:20.449060+0000","last_became_peered":"2023-11-04T05:28:20.449060+0000","last_unstale":"2023-11-04T05:28:31.743983+0000","last_undegraded":"2023-11-04T05:28:31.743983+0000","last_fullsized":"2023-11-04T05:28:31.743983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835007+0000","last_change":"2023-11-04T05:28:21.459298+0000","last_active":"2023-11-04T05:28:25.835007+0000","last_peered":"2023-11-04T05:28:25.835007+0000","last_clean":"2023-11-04T05:28:25.835007+0000","last_became_active":"2023-11-04T05:28:21.459046+0000","last_became_peered":"2023-11-04T05:28:21.459046+0000","last_unstale":"2023-11-04T05:28:25.835007+0000","last_undegraded":"2023-11-04T05:28:25.835007+0000","last_fullsized":"2023-11-04T05:28:25.835007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836097+0000","last_change":"2023-11-04T05:28:25.853324+0000","last_active":"2023-11-04T05:28:26.836097+0000","last_peered":"2023-11-04T05:28:26.836097+0000","last_clean":"2023-11-04T05:28:26.836097+0000","last_became_active":"2023-11-04T05:28:25.852913+0000","last_became_peered":"2023-11-04T05:28:25.852913+0000","last_unstale":"2023-11-04T05:28:26.836097+0000","last_undegraded":"2023-11-04T05:28:26.836097+0000","last_fullsized":"2023-11-04T05:28:26.836097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842933+0000","last_change":"2023-11-04T05:28:20.451652+0000","last_active":"2023-11-04T05:28:25.842933+0000","last_peered":"2023-11-04T05:28:25.842933+0000","last_clean":"2023-11-04T05:28:25.842933+0000","last_became_active":"2023-11-04T05:28:20.451415+0000","last_became_peered":"2023-11-04T05:28:20.451415+0000","last_unstale":"2023-11-04T05:28:25.842933+0000","last_undegraded":"2023-11-04T05:28:25.842933+0000","last_fullsized":"2023-11-04T05:28:25.842933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.132028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842574+0000","last_change":"2023-11-04T05:28:21.454210+0000","last_active":"2023-11-04T05:28:25.842574+0000","last_peered":"2023-11-04T05:28:25.842574+0000","last_clean":"2023-11-04T05:28:25.842574+0000","last_became_active":"2023-11-04T05:28:21.453894+0000","last_became_peered":"2023-11-04T05:28:21.453894+0000","last_unstale":"2023-11-04T05:28:25.842574+0000","last_undegraded":"2023-11-04T05:28:25.842574+0000","last_fullsized":"2023-11-04T05:28:25.842574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:28:59.983010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220347+0000","last_change":"2023-11-04T05:28:25.864286+0000","last_active":"2023-11-04T05:28:28.220347+0000","last_peered":"2023-11-04T05:28:28.220347+0000","last_clean":"2023-11-04T05:28:28.220347+0000","last_became_active":"2023-11-04T05:28:25.863831+0000","last_became_peered":"2023-11-04T05:28:25.863831+0000","last_unstale":"2023-11-04T05:28:28.220347+0000","last_undegraded":"2023-11-04T05:28:28.220347+0000","last_fullsized":"2023-11-04T05:28:28.220347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:40:57.121972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839219+0000","last_change":"2023-11-04T05:28:20.456235+0000","last_active":"2023-11-04T05:28:25.839219+0000","last_peered":"2023-11-04T05:28:25.839219+0000","last_clean":"2023-11-04T05:28:25.839219+0000","last_became_active":"2023-11-04T05:28:20.455992+0000","last_became_peered":"2023-11-04T05:28:20.455992+0000","last_unstale":"2023-11-04T05:28:25.839219+0000","last_undegraded":"2023-11-04T05:28:25.839219+0000","last_fullsized":"2023-11-04T05:28:25.839219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834802+0000","last_change":"2023-11-04T05:28:21.460351+0000","last_active":"2023-11-04T05:28:25.834802+0000","last_peered":"2023-11-04T05:28:25.834802+0000","last_clean":"2023-11-04T05:28:25.834802+0000","last_became_active":"2023-11-04T05:28:21.459737+0000","last_became_peered":"2023-11-04T05:28:21.459737+0000","last_unstale":"2023-11-04T05:28:25.834802+0000","last_undegraded":"2023-11-04T05:28:25.834802+0000","last_fullsized":"2023-11-04T05:28:25.834802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220525+0000","last_change":"2023-11-04T05:28:25.868351+0000","last_active":"2023-11-04T05:28:28.220525+0000","last_peered":"2023-11-04T05:28:28.220525+0000","last_clean":"2023-11-04T05:28:28.220525+0000","last_became_active":"2023-11-04T05:28:25.867909+0000","last_became_peered":"2023-11-04T05:28:25.867909+0000","last_unstale":"2023-11-04T05:28:28.220525+0000","last_undegraded":"2023-11-04T05:28:28.220525+0000","last_fullsized":"2023-11-04T05:28:28.220525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:08:06.813956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833857+0000","last_change":"2023-11-04T05:28:20.447033+0000","last_active":"2023-11-04T05:28:25.833857+0000","last_peered":"2023-11-04T05:28:25.833857+0000","last_clean":"2023-11-04T05:28:25.833857+0000","last_became_active":"2023-11-04T05:28:20.446767+0000","last_became_peered":"2023-11-04T05:28:20.446767+0000","last_unstale":"2023-11-04T05:28:25.833857+0000","last_undegraded":"2023-11-04T05:28:25.833857+0000","last_fullsized":"2023-11-04T05:28:25.833857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836084+0000","last_change":"2023-11-04T05:28:21.457261+0000","last_active":"2023-11-04T05:28:25.836084+0000","last_peered":"2023-11-04T05:28:25.836084+0000","last_clean":"2023-11-04T05:28:25.836084+0000","last_became_active":"2023-11-04T05:28:21.456982+0000","last_became_peered":"2023-11-04T05:28:21.456982+0000","last_unstale":"2023-11-04T05:28:25.836084+0000","last_undegraded":"2023-11-04T05:28:25.836084+0000","last_fullsized":"2023-11-04T05:28:25.836084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919838+0000","last_change":"2023-11-04T05:28:25.854585+0000","last_active":"2023-11-04T05:28:27.919838+0000","last_peered":"2023-11-04T05:28:27.919838+0000","last_clean":"2023-11-04T05:28:27.919838+0000","last_became_active":"2023-11-04T05:28:25.854151+0000","last_became_peered":"2023-11-04T05:28:25.854151+0000","last_unstale":"2023-11-04T05:28:27.919838+0000","last_undegraded":"2023-11-04T05:28:27.919838+0000","last_fullsized":"2023-11-04T05:28:27.919838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:05:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.844868+0000","last_change":"2023-11-04T05:28:20.448118+0000","last_active":"2023-11-04T05:28:25.844868+0000","last_peered":"2023-11-04T05:28:25.844868+0000","last_clean":"2023-11-04T05:28:25.844868+0000","last_became_active":"2023-11-04T05:28:20.447780+0000","last_became_peered":"2023-11-04T05:28:20.447780+0000","last_unstale":"2023-11-04T05:28:25.844868+0000","last_undegraded":"2023-11-04T05:28:25.844868+0000","last_fullsized":"2023-11-04T05:28:25.844868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:45:52.143624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834466+0000","last_change":"2023-11-04T05:28:21.458858+0000","last_active":"2023-11-04T05:28:25.834466+0000","last_peered":"2023-11-04T05:28:25.834466+0000","last_clean":"2023-11-04T05:28:25.834466+0000","last_became_active":"2023-11-04T05:28:21.458577+0000","last_became_peered":"2023-11-04T05:28:21.458577+0000","last_unstale":"2023-11-04T05:28:25.834466+0000","last_undegraded":"2023-11-04T05:28:25.834466+0000","last_fullsized":"2023-11-04T05:28:25.834466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742613+0000","last_change":"2023-11-04T05:28:25.865986+0000","last_active":"2023-11-04T05:28:28.742613+0000","last_peered":"2023-11-04T05:28:28.742613+0000","last_clean":"2023-11-04T05:28:28.742613+0000","last_became_active":"2023-11-04T05:28:25.865476+0000","last_became_peered":"2023-11-04T05:28:25.865476+0000","last_unstale":"2023-11-04T05:28:28.742613+0000","last_undegraded":"2023-11-04T05:28:28.742613+0000","last_fullsized":"2023-11-04T05:28:28.742613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:23.545163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842973+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.842973+0000","last_peered":"2023-11-04T05:28:25.842973+0000","last_clean":"2023-11-04T05:28:25.842973+0000","last_became_active":"2023-11-04T05:28:20.451353+0000","last_became_peered":"2023-11-04T05:28:20.451353+0000","last_unstale":"2023-11-04T05:28:25.842973+0000","last_undegraded":"2023-11-04T05:28:25.842973+0000","last_fullsized":"2023-11-04T05:28:25.842973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:11:52.224724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835325+0000","last_change":"2023-11-04T05:28:21.460107+0000","last_active":"2023-11-04T05:28:25.835325+0000","last_peered":"2023-11-04T05:28:25.835325+0000","last_clean":"2023-11-04T05:28:25.835325+0000","last_became_active":"2023-11-04T05:28:21.458229+0000","last_became_peered":"2023-11-04T05:28:21.458229+0000","last_unstale":"2023-11-04T05:28:25.835325+0000","last_undegraded":"2023-11-04T05:28:25.835325+0000","last_fullsized":"2023-11-04T05:28:25.835325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832264+0000","last_change":"2023-11-04T05:28:25.863802+0000","last_active":"2023-11-04T05:28:28.832264+0000","last_peered":"2023-11-04T05:28:28.832264+0000","last_clean":"2023-11-04T05:28:28.832264+0000","last_became_active":"2023-11-04T05:28:25.863268+0000","last_became_peered":"2023-11-04T05:28:25.863268+0000","last_unstale":"2023-11-04T05:28:28.832264+0000","last_undegraded":"2023-11-04T05:28:28.832264+0000","last_fullsized":"2023-11-04T05:28:28.832264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:30.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862531+0000","last_change":"2023-11-04T05:28:20.453776+0000","last_active":"2023-11-04T05:28:25.862531+0000","last_peered":"2023-11-04T05:28:25.862531+0000","last_clean":"2023-11-04T05:28:25.862531+0000","last_became_active":"2023-11-04T05:28:20.453564+0000","last_became_peered":"2023-11-04T05:28:20.453564+0000","last_unstale":"2023-11-04T05:28:25.862531+0000","last_undegraded":"2023-11-04T05:28:25.862531+0000","last_fullsized":"2023-11-04T05:28:25.862531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842239+0000","last_change":"2023-11-04T05:28:21.463249+0000","last_active":"2023-11-04T05:28:25.842239+0000","last_peered":"2023-11-04T05:28:25.842239+0000","last_clean":"2023-11-04T05:28:25.842239+0000","last_became_active":"2023-11-04T05:28:21.462964+0000","last_became_peered":"2023-11-04T05:28:21.462964+0000","last_unstale":"2023-11-04T05:28:25.842239+0000","last_undegraded":"2023-11-04T05:28:25.842239+0000","last_fullsized":"2023-11-04T05:28:25.842239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832100+0000","last_change":"2023-11-04T05:28:25.866264+0000","last_active":"2023-11-04T05:28:28.832100+0000","last_peered":"2023-11-04T05:28:28.832100+0000","last_clean":"2023-11-04T05:28:28.832100+0000","last_became_active":"2023-11-04T05:28:25.865946+0000","last_became_peered":"2023-11-04T05:28:25.865946+0000","last_unstale":"2023-11-04T05:28:28.832100+0000","last_undegraded":"2023-11-04T05:28:28.832100+0000","last_fullsized":"2023-11-04T05:28:28.832100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839118+0000","last_change":"2023-11-04T05:28:20.449195+0000","last_active":"2023-11-04T05:28:25.839118+0000","last_peered":"2023-11-04T05:28:25.839118+0000","last_clean":"2023-11-04T05:28:25.839118+0000","last_became_active":"2023-11-04T05:28:20.448862+0000","last_became_peered":"2023-11-04T05:28:20.448862+0000","last_unstale":"2023-11-04T05:28:25.839118+0000","last_undegraded":"2023-11-04T05:28:25.839118+0000","last_fullsized":"2023-11-04T05:28:25.839118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:32.638827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843378+0000","last_change":"2023-11-04T05:28:21.464603+0000","last_active":"2023-11-04T05:28:25.843378+0000","last_peered":"2023-11-04T05:28:25.843378+0000","last_clean":"2023-11-04T05:28:25.843378+0000","last_became_active":"2023-11-04T05:28:21.464337+0000","last_became_peered":"2023-11-04T05:28:21.464337+0000","last_unstale":"2023-11-04T05:28:25.843378+0000","last_undegraded":"2023-11-04T05:28:25.843378+0000","last_fullsized":"2023-11-04T05:28:25.843378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:44.443386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832325+0000","last_change":"2023-11-04T05:28:25.864109+0000","last_active":"2023-11-04T05:28:28.832325+0000","last_peered":"2023-11-04T05:28:28.832325+0000","last_clean":"2023-11-04T05:28:28.832325+0000","last_became_active":"2023-11-04T05:28:25.863293+0000","last_became_peered":"2023-11-04T05:28:25.863293+0000","last_unstale":"2023-11-04T05:28:28.832325+0000","last_undegraded":"2023-11-04T05:28:28.832325+0000","last_fullsized":"2023-11-04T05:28:28.832325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:13.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857963+0000","last_change":"2023-11-04T05:28:21.448655+0000","last_active":"2023-11-04T05:28:25.857963+0000","last_peered":"2023-11-04T05:28:25.857963+0000","last_clean":"2023-11-04T05:28:25.857963+0000","last_became_active":"2023-11-04T05:28:21.448355+0000","last_became_peered":"2023-11-04T05:28:21.448355+0000","last_unstale":"2023-11-04T05:28:25.857963+0000","last_undegraded":"2023-11-04T05:28:25.857963+0000","last_fullsized":"2023-11-04T05:28:25.857963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845533+0000","last_change":"2023-11-04T05:28:20.453706+0000","last_active":"2023-11-04T05:28:25.845533+0000","last_peered":"2023-11-04T05:28:25.845533+0000","last_clean":"2023-11-04T05:28:25.845533+0000","last_became_active":"2023-11-04T05:28:20.453428+0000","last_became_peered":"2023-11-04T05:28:20.453428+0000","last_unstale":"2023-11-04T05:28:25.845533+0000","last_undegraded":"2023-11-04T05:28:25.845533+0000","last_fullsized":"2023-11-04T05:28:25.845533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:43.438258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838700+0000","last_change":"2023-11-04T05:28:25.852681+0000","last_active":"2023-11-04T05:28:26.838700+0000","last_peered":"2023-11-04T05:28:26.838700+0000","last_clean":"2023-11-04T05:28:26.838700+0000","last_became_active":"2023-11-04T05:28:25.852276+0000","last_became_peered":"2023-11-04T05:28:25.852276+0000","last_unstale":"2023-11-04T05:28:26.838700+0000","last_undegraded":"2023-11-04T05:28:26.838700+0000","last_fullsized":"2023-11-04T05:28:26.838700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:28.005917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840730+0000","last_change":"2023-11-04T05:28:21.466480+0000","last_active":"2023-11-04T05:28:25.840730+0000","last_peered":"2023-11-04T05:28:25.840730+0000","last_clean":"2023-11-04T05:28:25.840730+0000","last_became_active":"2023-11-04T05:28:21.466065+0000","last_became_peered":"2023-11-04T05:28:21.466065+0000","last_unstale":"2023-11-04T05:28:25.840730+0000","last_undegraded":"2023-11-04T05:28:25.840730+0000","last_fullsized":"2023-11-04T05:28:25.840730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.838807+0000","last_change":"2023-11-04T05:28:20.451336+0000","last_active":"2023-11-04T05:28:25.838807+0000","last_peered":"2023-11-04T05:28:25.838807+0000","last_clean":"2023-11-04T05:28:25.838807+0000","last_became_active":"2023-11-04T05:28:20.451000+0000","last_became_peered":"2023-11-04T05:28:20.451000+0000","last_unstale":"2023-11-04T05:28:25.838807+0000","last_undegraded":"2023-11-04T05:28:25.838807+0000","last_fullsized":"2023-11-04T05:28:25.838807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:46.798012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834100+0000","last_change":"2023-11-04T05:28:11.829985+0000","last_active":"2023-11-04T05:28:25.834100+0000","last_peered":"2023-11-04T05:28:25.834100+0000","last_clean":"2023-11-04T05:28:25.834100+0000","last_became_active":"2023-11-04T05:28:11.829705+0000","last_became_peered":"2023-11-04T05:28:11.829705+0000","last_unstale":"2023-11-04T05:28:25.834100+0000","last_undegraded":"2023-11-04T05:28:25.834100+0000","last_fullsized":"2023-11-04T05:28:25.834100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_clean_scrub_stamp":"2023-11-04T05:28:10.378947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:46:15.100605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919902+0000","last_change":"2023-11-04T05:28:25.865855+0000","last_active":"2023-11-04T05:28:27.919902+0000","last_peered":"2023-11-04T05:28:27.919902+0000","last_clean":"2023-11-04T05:28:27.919902+0000","last_became_active":"2023-11-04T05:28:25.865538+0000","last_became_peered":"2023-11-04T05:28:25.865538+0000","last_unstale":"2023-11-04T05:28:27.919902+0000","last_undegraded":"2023-11-04T05:28:27.919902+0000","last_fullsized":"2023-11-04T05:28:27.919902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:24:58.810725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842382+0000","last_change":"2023-11-04T05:28:21.461543+0000","last_active":"2023-11-04T05:28:25.842382+0000","last_peered":"2023-11-04T05:28:25.842382+0000","last_clean":"2023-11-04T05:28:25.842382+0000","last_became_active":"2023-11-04T05:28:21.461195+0000","last_became_peered":"2023-11-04T05:28:21.461195+0000","last_unstale":"2023-11-04T05:28:25.842382+0000","last_undegraded":"2023-11-04T05:28:25.842382+0000","last_fullsized":"2023-11-04T05:28:25.842382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:28:07.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834194+0000","last_change":"2023-11-04T05:28:20.450213+0000","last_active":"2023-11-04T05:28:25.834194+0000","last_peered":"2023-11-04T05:28:25.834194+0000","last_clean":"2023-11-04T05:28:25.834194+0000","last_became_active":"2023-11-04T05:28:20.449957+0000","last_became_peered":"2023-11-04T05:28:20.449957+0000","last_unstale":"2023-11-04T05:28:25.834194+0000","last_undegraded":"2023-11-04T05:28:25.834194+0000","last_fullsized":"2023-11-04T05:28:25.834194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:30:12.799059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919653+0000","last_change":"2023-11-04T05:28:25.861935+0000","last_active":"2023-11-04T05:28:27.919653+0000","last_peered":"2023-11-04T05:28:27.919653+0000","last_clean":"2023-11-04T05:28:27.919653+0000","last_became_active":"2023-11-04T05:28:25.861471+0000","last_became_peered":"2023-11-04T05:28:25.861471+0000","last_unstale":"2023-11-04T05:28:27.919653+0000","last_undegraded":"2023-11-04T05:28:27.919653+0000","last_fullsized":"2023-11-04T05:28:27.919653+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:53:47.409110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834605+0000","last_change":"2023-11-04T05:28:20.455821+0000","last_active":"2023-11-04T05:28:25.834605+0000","last_peered":"2023-11-04T05:28:25.834605+0000","last_clean":"2023-11-04T05:28:25.834605+0000","last_became_active":"2023-11-04T05:28:20.455538+0000","last_became_peered":"2023-11-04T05:28:20.455538+0000","last_unstale":"2023-11-04T05:28:25.834605+0000","last_undegraded":"2023-11-04T05:28:25.834605+0000","last_fullsized":"2023-11-04T05:28:25.834605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862653+0000","last_change":"2023-11-04T05:28:21.446022+0000","last_active":"2023-11-04T05:28:25.862653+0000","last_peered":"2023-11-04T05:28:25.862653+0000","last_clean":"2023-11-04T05:28:25.862653+0000","last_became_active":"2023-11-04T05:28:21.445727+0000","last_became_peered":"2023-11-04T05:28:21.445727+0000","last_unstale":"2023-11-04T05:28:25.862653+0000","last_undegraded":"2023-11-04T05:28:25.862653+0000","last_fullsized":"2023-11-04T05:28:25.862653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219703+0000","last_change":"2023-11-04T05:28:25.860350+0000","last_active":"2023-11-04T05:28:28.219703+0000","last_peered":"2023-11-04T05:28:28.219703+0000","last_clean":"2023-11-04T05:28:28.219703+0000","last_became_active":"2023-11-04T05:28:25.859724+0000","last_became_peered":"2023-11-04T05:28:25.859724+0000","last_unstale":"2023-11-04T05:28:28.219703+0000","last_undegraded":"2023-11-04T05:28:28.219703+0000","last_fullsized":"2023-11-04T05:28:28.219703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:33.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.356335+0000","last_change":"2023-11-04T05:28:20.449071+0000","last_active":"2023-11-04T05:28:26.356335+0000","last_peered":"2023-11-04T05:28:26.356335+0000","last_clean":"2023-11-04T05:28:26.356335+0000","last_became_active":"2023-11-04T05:28:20.448778+0000","last_became_peered":"2023-11-04T05:28:20.448778+0000","last_unstale":"2023-11-04T05:28:26.356335+0000","last_undegraded":"2023-11-04T05:28:26.356335+0000","last_fullsized":"2023-11-04T05:28:26.356335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840260+0000","last_change":"2023-11-04T05:28:21.444332+0000","last_active":"2023-11-04T05:28:25.840260+0000","last_peered":"2023-11-04T05:28:25.840260+0000","last_clean":"2023-11-04T05:28:25.840260+0000","last_became_active":"2023-11-04T05:28:21.444027+0000","last_became_peered":"2023-11-04T05:28:21.444027+0000","last_unstale":"2023-11-04T05:28:25.840260+0000","last_undegraded":"2023-11-04T05:28:25.840260+0000","last_fullsized":"2023-11-04T05:28:25.840260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742687+0000","last_change":"2023-11-04T05:28:25.846982+0000","last_active":"2023-11-04T05:28:28.742687+0000","last_peered":"2023-11-04T05:28:28.742687+0000","last_clean":"2023-11-04T05:28:28.742687+0000","last_became_active":"2023-11-04T05:28:25.846562+0000","last_became_peered":"2023-11-04T05:28:25.846562+0000","last_unstale":"2023-11-04T05:28:28.742687+0000","last_undegraded":"2023-11-04T05:28:28.742687+0000","last_fullsized":"2023-11-04T05:28:28.742687+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:52.840764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834969+0000","last_change":"2023-11-04T05:28:20.447063+0000","last_active":"2023-11-04T05:28:25.834969+0000","last_peered":"2023-11-04T05:28:25.834969+0000","last_clean":"2023-11-04T05:28:25.834969+0000","last_became_active":"2023-11-04T05:28:20.446833+0000","last_became_peered":"2023-11-04T05:28:20.446833+0000","last_unstale":"2023-11-04T05:28:25.834969+0000","last_undegraded":"2023-11-04T05:28:25.834969+0000","last_fullsized":"2023-11-04T05:28:25.834969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842874+0000","last_change":"2023-11-04T05:28:21.456075+0000","last_active":"2023-11-04T05:28:25.842874+0000","last_peered":"2023-11-04T05:28:25.842874+0000","last_clean":"2023-11-04T05:28:25.842874+0000","last_became_active":"2023-11-04T05:28:21.455801+0000","last_became_peered":"2023-11-04T05:28:21.455801+0000","last_unstale":"2023-11-04T05:28:25.842874+0000","last_undegraded":"2023-11-04T05:28:25.842874+0000","last_fullsized":"2023-11-04T05:28:25.842874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:48.444983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743366+0000","last_change":"2023-11-04T05:28:25.864053+0000","last_active":"2023-11-04T05:28:28.743366+0000","last_peered":"2023-11-04T05:28:28.743366+0000","last_clean":"2023-11-04T05:28:28.743366+0000","last_became_active":"2023-11-04T05:28:25.863669+0000","last_became_peered":"2023-11-04T05:28:25.863669+0000","last_unstale":"2023-11-04T05:28:28.743366+0000","last_undegraded":"2023-11-04T05:28:28.743366+0000","last_fullsized":"2023-11-04T05:28:28.743366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:42:30.660108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833998+0000","last_change":"2023-11-04T05:28:20.453827+0000","last_active":"2023-11-04T05:28:25.833998+0000","last_peered":"2023-11-04T05:28:25.833998+0000","last_clean":"2023-11-04T05:28:25.833998+0000","last_became_active":"2023-11-04T05:28:20.453576+0000","last_became_peered":"2023-11-04T05:28:20.453576+0000","last_unstale":"2023-11-04T05:28:25.833998+0000","last_undegraded":"2023-11-04T05:28:25.833998+0000","last_fullsized":"2023-11-04T05:28:25.833998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840329+0000","last_change":"2023-11-04T05:28:21.458086+0000","last_active":"2023-11-04T05:28:25.840329+0000","last_peered":"2023-11-04T05:28:25.840329+0000","last_clean":"2023-11-04T05:28:25.840329+0000","last_became_active":"2023-11-04T05:28:21.457739+0000","last_became_peered":"2023-11-04T05:28:21.457739+0000","last_unstale":"2023-11-04T05:28:25.840329+0000","last_undegraded":"2023-11-04T05:28:25.840329+0000","last_fullsized":"2023-11-04T05:28:25.840329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:45.088645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832505+0000","last_change":"2023-11-04T05:28:25.864371+0000","last_active":"2023-11-04T05:28:28.832505+0000","last_peered":"2023-11-04T05:28:28.832505+0000","last_clean":"2023-11-04T05:28:28.832505+0000","last_became_active":"2023-11-04T05:28:25.863374+0000","last_became_peered":"2023-11-04T05:28:25.863374+0000","last_unstale":"2023-11-04T05:28:28.832505+0000","last_undegraded":"2023-11-04T05:28:28.832505+0000","last_fullsized":"2023-11-04T05:28:28.832505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:58.207114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857793+0000","last_change":"2023-11-04T05:28:20.450039+0000","last_active":"2023-11-04T05:28:25.857793+0000","last_peered":"2023-11-04T05:28:25.857793+0000","last_clean":"2023-11-04T05:28:25.857793+0000","last_became_active":"2023-11-04T05:28:20.449761+0000","last_became_peered":"2023-11-04T05:28:20.449761+0000","last_unstale":"2023-11-04T05:28:25.857793+0000","last_undegraded":"2023-11-04T05:28:25.857793+0000","last_fullsized":"2023-11-04T05:28:25.857793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841634+0000","last_change":"2023-11-04T05:28:21.450153+0000","last_active":"2023-11-04T05:28:25.841634+0000","last_peered":"2023-11-04T05:28:25.841634+0000","last_clean":"2023-11-04T05:28:25.841634+0000","last_became_active":"2023-11-04T05:28:21.449878+0000","last_became_peered":"2023-11-04T05:28:21.449878+0000","last_unstale":"2023-11-04T05:28:25.841634+0000","last_undegraded":"2023-11-04T05:28:25.841634+0000","last_fullsized":"2023-11-04T05:28:25.841634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743302+0000","last_change":"2023-11-04T05:28:25.864439+0000","last_active":"2023-11-04T05:28:28.743302+0000","last_peered":"2023-11-04T05:28:28.743302+0000","last_clean":"2023-11-04T05:28:28.743302+0000","last_became_active":"2023-11-04T05:28:25.863909+0000","last_became_peered":"2023-11-04T05:28:25.863909+0000","last_unstale":"2023-11-04T05:28:28.743302+0000","last_undegraded":"2023-11-04T05:28:28.743302+0000","last_fullsized":"2023-11-04T05:28:28.743302+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:00:50.267120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858628+0000","last_change":"2023-11-04T05:28:20.450297+0000","last_active":"2023-11-04T05:28:25.858628+0000","last_peered":"2023-11-04T05:28:25.858628+0000","last_clean":"2023-11-04T05:28:25.858628+0000","last_became_active":"2023-11-04T05:28:20.450017+0000","last_became_peered":"2023-11-04T05:28:20.450017+0000","last_unstale":"2023-11-04T05:28:25.858628+0000","last_undegraded":"2023-11-04T05:28:25.858628+0000","last_fullsized":"2023-11-04T05:28:25.858628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:14:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840528+0000","last_change":"2023-11-04T05:28:21.457231+0000","last_active":"2023-11-04T05:28:25.840528+0000","last_peered":"2023-11-04T05:28:25.840528+0000","last_clean":"2023-11-04T05:28:25.840528+0000","last_became_active":"2023-11-04T05:28:21.456966+0000","last_became_peered":"2023-11-04T05:28:21.456966+0000","last_unstale":"2023-11-04T05:28:25.840528+0000","last_undegraded":"2023-11-04T05:28:25.840528+0000","last_fullsized":"2023-11-04T05:28:25.840528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743383+0000","last_change":"2023-11-04T05:28:25.864991+0000","last_active":"2023-11-04T05:28:28.743383+0000","last_peered":"2023-11-04T05:28:28.743383+0000","last_clean":"2023-11-04T05:28:28.743383+0000","last_became_active":"2023-11-04T05:28:25.864122+0000","last_became_peered":"2023-11-04T05:28:25.864122+0000","last_unstale":"2023-11-04T05:28:28.743383+0000","last_undegraded":"2023-11-04T05:28:28.743383+0000","last_fullsized":"2023-11-04T05:28:28.743383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:28:31.685109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834888+0000","last_change":"2023-11-04T05:28:20.449101+0000","last_active":"2023-11-04T05:28:25.834888+0000","last_peered":"2023-11-04T05:28:25.834888+0000","last_clean":"2023-11-04T05:28:25.834888+0000","last_became_active":"2023-11-04T05:28:20.448809+0000","last_became_peered":"2023-11-04T05:28:20.448809+0000","last_unstale":"2023-11-04T05:28:25.834888+0000","last_undegraded":"2023-11-04T05:28:25.834888+0000","last_fullsized":"2023-11-04T05:28:25.834888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834773+0000","last_change":"2023-11-04T05:28:21.456943+0000","last_active":"2023-11-04T05:28:25.834773+0000","last_peered":"2023-11-04T05:28:25.834773+0000","last_clean":"2023-11-04T05:28:25.834773+0000","last_became_active":"2023-11-04T05:28:21.456658+0000","last_became_peered":"2023-11-04T05:28:21.456658+0000","last_unstale":"2023-11-04T05:28:25.834773+0000","last_undegraded":"2023-11-04T05:28:25.834773+0000","last_fullsized":"2023-11-04T05:28:25.834773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838509+0000","last_change":"2023-11-04T05:28:25.865380+0000","last_active":"2023-11-04T05:28:26.838509+0000","last_peered":"2023-11-04T05:28:26.838509+0000","last_clean":"2023-11-04T05:28:26.838509+0000","last_became_active":"2023-11-04T05:28:25.865014+0000","last_became_peered":"2023-11-04T05:28:25.865014+0000","last_unstale":"2023-11-04T05:28:26.838509+0000","last_undegraded":"2023-11-04T05:28:26.838509+0000","last_fullsized":"2023-11-04T05:28:26.838509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:08.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846470+0000","last_change":"2023-11-04T05:28:20.451801+0000","last_active":"2023-11-04T05:28:25.846470+0000","last_peered":"2023-11-04T05:28:25.846470+0000","last_clean":"2023-11-04T05:28:25.846470+0000","last_became_active":"2023-11-04T05:28:20.451470+0000","last_became_peered":"2023-11-04T05:28:20.451470+0000","last_unstale":"2023-11-04T05:28:25.846470+0000","last_undegraded":"2023-11-04T05:28:25.846470+0000","last_fullsized":"2023-11-04T05:28:25.846470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:26:53.409369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834801+0000","last_change":"2023-11-04T05:28:21.465901+0000","last_active":"2023-11-04T05:28:25.834801+0000","last_peered":"2023-11-04T05:28:25.834801+0000","last_clean":"2023-11-04T05:28:25.834801+0000","last_became_active":"2023-11-04T05:28:21.465612+0000","last_became_peered":"2023-11-04T05:28:21.465612+0000","last_unstale":"2023-11-04T05:28:25.834801+0000","last_undegraded":"2023-11-04T05:28:25.834801+0000","last_fullsized":"2023-11-04T05:28:25.834801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919553+0000","last_change":"2023-11-04T05:28:25.858664+0000","last_active":"2023-11-04T05:28:27.919553+0000","last_peered":"2023-11-04T05:28:27.919553+0000","last_clean":"2023-11-04T05:28:27.919553+0000","last_became_active":"2023-11-04T05:28:25.858208+0000","last_became_peered":"2023-11-04T05:28:25.858208+0000","last_unstale":"2023-11-04T05:28:27.919553+0000","last_undegraded":"2023-11-04T05:28:27.919553+0000","last_fullsized":"2023-11-04T05:28:27.919553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:41:26.004032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858515+0000","last_change":"2023-11-04T05:28:20.443824+0000","last_active":"2023-11-04T05:28:25.858515+0000","last_peered":"2023-11-04T05:28:25.858515+0000","last_clean":"2023-11-04T05:28:25.858515+0000","last_became_active":"2023-11-04T05:28:20.443479+0000","last_became_peered":"2023-11-04T05:28:20.443479+0000","last_unstale":"2023-11-04T05:28:25.858515+0000","last_undegraded":"2023-11-04T05:28:25.858515+0000","last_fullsized":"2023-11-04T05:28:25.858515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840613+0000","last_change":"2023-11-04T05:28:21.463699+0000","last_active":"2023-11-04T05:28:25.840613+0000","last_peered":"2023-11-04T05:28:25.840613+0000","last_clean":"2023-11-04T05:28:25.840613+0000","last_became_active":"2023-11-04T05:28:21.462308+0000","last_became_peered":"2023-11-04T05:28:21.462308+0000","last_unstale":"2023-11-04T05:28:25.840613+0000","last_undegraded":"2023-11-04T05:28:25.840613+0000","last_fullsized":"2023-11-04T05:28:25.840613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836473+0000","last_change":"2023-11-04T05:28:25.849562+0000","last_active":"2023-11-04T05:28:26.836473+0000","last_peered":"2023-11-04T05:28:26.836473+0000","last_clean":"2023-11-04T05:28:26.836473+0000","last_became_active":"2023-11-04T05:28:25.849142+0000","last_became_peered":"2023-11-04T05:28:25.849142+0000","last_unstale":"2023-11-04T05:28:26.836473+0000","last_undegraded":"2023-11-04T05:28:26.836473+0000","last_fullsized":"2023-11-04T05:28:26.836473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846568+0000","last_change":"2023-11-04T05:28:20.455376+0000","last_active":"2023-11-04T05:28:25.846568+0000","last_peered":"2023-11-04T05:28:25.846568+0000","last_clean":"2023-11-04T05:28:25.846568+0000","last_became_active":"2023-11-04T05:28:20.455069+0000","last_became_peered":"2023-11-04T05:28:20.455069+0000","last_unstale":"2023-11-04T05:28:25.846568+0000","last_undegraded":"2023-11-04T05:28:25.846568+0000","last_fullsized":"2023-11-04T05:28:25.846568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:44.083516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835570+0000","last_change":"2023-11-04T05:28:21.459762+0000","last_active":"2023-11-04T05:28:25.835570+0000","last_peered":"2023-11-04T05:28:25.835570+0000","last_clean":"2023-11-04T05:28:25.835570+0000","last_became_active":"2023-11-04T05:28:21.459488+0000","last_became_peered":"2023-11-04T05:28:21.459488+0000","last_unstale":"2023-11-04T05:28:25.835570+0000","last_undegraded":"2023-11-04T05:28:25.835570+0000","last_fullsized":"2023-11-04T05:28:25.835570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836085+0000","last_change":"2023-11-04T05:28:21.458229+0000","last_active":"2023-11-04T05:28:25.836085+0000","last_peered":"2023-11-04T05:28:25.836085+0000","last_clean":"2023-11-04T05:28:25.836085+0000","last_became_active":"2023-11-04T05:28:21.457918+0000","last_became_peered":"2023-11-04T05:28:21.457918+0000","last_unstale":"2023-11-04T05:28:25.836085+0000","last_undegraded":"2023-11-04T05:28:25.836085+0000","last_fullsized":"2023-11-04T05:28:25.836085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845334+0000","last_change":"2023-11-04T05:28:21.464900+0000","last_active":"2023-11-04T05:28:25.845334+0000","last_peered":"2023-11-04T05:28:25.845334+0000","last_clean":"2023-11-04T05:28:25.845334+0000","last_became_active":"2023-11-04T05:28:21.463859+0000","last_became_peered":"2023-11-04T05:28:21.463859+0000","last_unstale":"2023-11-04T05:28:25.845334+0000","last_undegraded":"2023-11-04T05:28:25.845334+0000","last_fullsized":"2023-11-04T05:28:25.845334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839310+0000","last_change":"2023-11-04T05:28:21.466114+0000","last_active":"2023-11-04T05:28:25.839310+0000","last_peered":"2023-11-04T05:28:25.839310+0000","last_clean":"2023-11-04T05:28:25.839310+0000","last_became_active":"2023-11-04T05:28:21.465547+0000","last_became_peered":"2023-11-04T05:28:21.465547+0000","last_unstale":"2023-11-04T05:28:25.839310+0000","last_undegraded":"2023-11-04T05:28:25.839310+0000","last_fullsized":"2023-11-04T05:28:25.839310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842759+0000","last_change":"2023-11-04T05:28:21.456527+0000","last_active":"2023-11-04T05:28:25.842759+0000","last_peered":"2023-11-04T05:28:25.842759+0000","last_clean":"2023-11-04T05:28:25.842759+0000","last_became_active":"2023-11-04T05:28:21.456157+0000","last_became_peered":"2023-11-04T05:28:21.456157+0000","last_unstale":"2023-11-04T05:28:25.842759+0000","last_undegraded":"2023-11-04T05:28:25.842759+0000","last_fullsized":"2023-11-04T05:28:25.842759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:19.149382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840466+0000","last_change":"2023-11-04T05:28:21.460439+0000","last_active":"2023-11-04T05:28:25.840466+0000","last_peered":"2023-11-04T05:28:25.840466+0000","last_clean":"2023-11-04T05:28:25.840466+0000","last_became_active":"2023-11-04T05:28:21.460146+0000","last_became_peered":"2023-11-04T05:28:21.460146+0000","last_unstale":"2023-11-04T05:28:25.840466+0000","last_undegraded":"2023-11-04T05:28:25.840466+0000","last_fullsized":"2023-11-04T05:28:25.840466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858719+0000","last_change":"2023-11-04T05:28:21.447651+0000","last_active":"2023-11-04T05:28:25.858719+0000","last_peered":"2023-11-04T05:28:25.858719+0000","last_clean":"2023-11-04T05:28:25.858719+0000","last_became_active":"2023-11-04T05:28:21.447381+0000","last_became_peered":"2023-11-04T05:28:21.447381+0000","last_unstale":"2023-11-04T05:28:25.858719+0000","last_undegraded":"2023-11-04T05:28:25.858719+0000","last_fullsized":"2023-11-04T05:28:25.858719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835914+0000","last_change":"2023-11-04T05:28:21.443196+0000","last_active":"2023-11-04T05:28:25.835914+0000","last_peered":"2023-11-04T05:28:25.835914+0000","last_clean":"2023-11-04T05:28:25.835914+0000","last_became_active":"2023-11-04T05:28:21.442860+0000","last_became_peered":"2023-11-04T05:28:21.442860+0000","last_unstale":"2023-11-04T05:28:25.835914+0000","last_undegraded":"2023-11-04T05:28:25.835914+0000","last_fullsized":"2023-11-04T05:28:25.835914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841180+0000","last_change":"2023-11-04T05:28:21.455732+0000","last_active":"2023-11-04T05:28:25.841180+0000","last_peered":"2023-11-04T05:28:25.841180+0000","last_clean":"2023-11-04T05:28:25.841180+0000","last_became_active":"2023-11-04T05:28:21.455434+0000","last_became_peered":"2023-11-04T05:28:21.455434+0000","last_unstale":"2023-11-04T05:28:25.841180+0000","last_undegraded":"2023-11-04T05:28:25.841180+0000","last_fullsized":"2023-11-04T05:28:25.841180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839745+0000","last_change":"2023-11-04T05:28:21.457789+0000","last_active":"2023-11-04T05:28:25.839745+0000","last_peered":"2023-11-04T05:28:25.839745+0000","last_clean":"2023-11-04T05:28:25.839745+0000","last_became_active":"2023-11-04T05:28:21.457521+0000","last_became_peered":"2023-11-04T05:28:21.457521+0000","last_unstale":"2023-11-04T05:28:25.839745+0000","last_undegraded":"2023-11-04T05:28:25.839745+0000","last_fullsized":"2023-11-04T05:28:25.839745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841996+0000","last_change":"2023-11-04T05:28:21.462501+0000","last_active":"2023-11-04T05:28:25.841996+0000","last_peered":"2023-11-04T05:28:25.841996+0000","last_clean":"2023-11-04T05:28:25.841996+0000","last_became_active":"2023-11-04T05:28:21.462147+0000","last_became_peered":"2023-11-04T05:28:21.462147+0000","last_unstale":"2023-11-04T05:28:25.841996+0000","last_undegraded":"2023-11-04T05:28:25.841996+0000","last_fullsized":"2023-11-04T05:28:25.841996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839367+0000","last_change":"2023-11-04T05:28:21.462044+0000","last_active":"2023-11-04T05:28:25.839367+0000","last_peered":"2023-11-04T05:28:25.839367+0000","last_clean":"2023-11-04T05:28:25.839367+0000","last_became_active":"2023-11-04T05:28:21.461816+0000","last_became_peered":"2023-11-04T05:28:21.461816+0000","last_unstale":"2023-11-04T05:28:25.839367+0000","last_undegraded":"2023-11-04T05:28:25.839367+0000","last_fullsized":"2023-11-04T05:28:25.839367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843103+0000","last_change":"2023-11-04T05:28:21.458256+0000","last_active":"2023-11-04T05:28:25.843103+0000","last_peered":"2023-11-04T05:28:25.843103+0000","last_clean":"2023-11-04T05:28:25.843103+0000","last_became_active":"2023-11-04T05:28:21.457944+0000","last_became_peered":"2023-11-04T05:28:21.457944+0000","last_unstale":"2023-11-04T05:28:25.843103+0000","last_undegraded":"2023-11-04T05:28:25.843103+0000","last_fullsized":"2023-11-04T05:28:25.843103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:41:28.397824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834927+0000","last_change":"2023-11-04T05:28:21.451553+0000","last_active":"2023-11-04T05:28:25.834927+0000","last_peered":"2023-11-04T05:28:25.834927+0000","last_clean":"2023-11-04T05:28:25.834927+0000","last_became_active":"2023-11-04T05:28:21.451276+0000","last_became_peered":"2023-11-04T05:28:21.451276+0000","last_unstale":"2023-11-04T05:28:25.834927+0000","last_undegraded":"2023-11-04T05:28:25.834927+0000","last_fullsized":"2023-11-04T05:28:25.834927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":697852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":697908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":107028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.221}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":106992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-04T05:28:46.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-04T05:28:46.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.195+0000 7f80da26f700 1 Processor -- start 2023-11-04T05:28:46.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.195+0000 7f80da26f700 1 -- start start 2023-11-04T05:28:46.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80da26f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80d40728b0 0x7f80d41688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80da26f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40cda00 con 0x7f80d406ab20 2023-11-04T05:28:46.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80da26f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40cdce0 con 0x7f80d40cc7a0 2023-11-04T05:28:46.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80da26f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40cf050 con 0x7f80d40728b0 2023-11-04T05:28:46.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80d880c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80d40728b0 0x7f80d41688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80d3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f80d406ab20 0x7f80d40cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57274/0 (socket says 172.21.15.3:57274) 2023-11-04T05:28:46.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.196+0000 7f80d3fff700 1 -- 172.21.15.3:0/1303081523 learned_addr learned my addr 172.21.15.3:0/1303081523 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:46.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d2ffd700 1 -- 172.21.15.3:0/1303081523 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514380009 0 0) 0x7f80d40cda00 con 0x7f80d406ab20 2023-11-04T05:28:46.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d880c700 1 -- 172.21.15.3:0/1303081523 >> v1:172.21.15.188:6789/0 conn(0x7f80d40cc7a0 legacy=0x7f80d406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:46.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d880c700 1 -- 172.21.15.3:0/1303081523 >> v1:172.21.15.3:6789/0 conn(0x7f80d406ab20 legacy=0x7f80d40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d880c700 1 -- 172.21.15.3:0/1303081523 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40720b0 con 0x7f80d40728b0 2023-11-04T05:28:46.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d880c700 1 --2- 172.21.15.3:0/1303081523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40728b0 0x7f80d41688e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009d90 tx=0x7f80cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2111da43b46354 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:46.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d2ffd700 1 -- 172.21.15.3:0/1303081523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00e070 con 0x7f80d40728b0 2023-11-04T05:28:46.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d2ffd700 1 -- 172.21.15.3:0/1303081523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc00bb90 con 0x7f80d40728b0 2023-11-04T05:28:46.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80d2ffd700 1 -- 172.21.15.3:0/1303081523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc003a20 con 0x7f80d40728b0 2023-11-04T05:28:46.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80da26f700 1 -- 172.21.15.3:0/1303081523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40728b0 msgr2=0x7f80d41688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80da26f700 1 --2- 172.21.15.3:0/1303081523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40728b0 0x7f80d41688e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009d90 tx=0x7f80cc00b330 comp rx=0 tx=0).stop 2023-11-04T05:28:46.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.197+0000 7f80da26f700 1 -- 172.21.15.3:0/1303081523 shutdown_connections 2023-11-04T05:28:46.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.198+0000 7f80da26f700 1 --2- 172.21.15.3:0/1303081523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40728b0 0x7f80d41688e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.198+0000 7f80da26f700 1 -- 172.21.15.3:0/1303081523 >> 172.21.15.3:0/1303081523 conn(0x7f80d406df90 msgr2=0x7f80d406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:46.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.198+0000 7f80da26f700 1 -- 172.21.15.3:0/1303081523 shutdown_connections 2023-11-04T05:28:46.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.198+0000 7f80da26f700 1 -- 172.21.15.3:0/1303081523 wait complete. 2023-11-04T05:28:46.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.199+0000 7f80da26f700 1 Processor -- start 2023-11-04T05:28:46.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.199+0000 7f80da26f700 1 -- start start 2023-11-04T05:28:46.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80da26f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80d406ab20 0x7f80d40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80da26f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d40728b0 0x7f80d41525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80da26f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 0x7f80d4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80da26f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40ae4f0 con 0x7f80d40cc7a0 2023-11-04T05:28:46.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80da26f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40af350 con 0x7f80d40728b0 2023-11-04T05:28:46.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80da26f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40ae740 con 0x7f80d406ab20 2023-11-04T05:28:46.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 0x7f80d4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 0x7f80d4157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51936/0 (socket says 172.21.15.3:51936) 2023-11-04T05:28:46.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 -- 172.21.15.3:0/1548660110 learned_addr learned my addr 172.21.15.3:0/1548660110 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:46.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d37fe700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d40728b0 0x7f80d41525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d3fff700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80d406ab20 0x7f80d40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 -- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80d406ab20 msgr2=0x7f80d40b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80d406ab20 0x7f80d40b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 -- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d40728b0 msgr2=0x7f80d41525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d40728b0 0x7f80d41525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d880c700 1 -- 172.21.15.3:0/1548660110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc009a70 con 0x7f80d40cc7a0 2023-11-04T05:28:46.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.200+0000 7f80d37fe700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d40728b0 0x7f80d41525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:46.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.201+0000 7f80d880c700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 0x7f80d4157c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f80d40d1640 tx=0x7f80cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:46.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.201+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc012630 con 0x7f80d40cc7a0 2023-11-04T05:28:46.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.201+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc0127b0 con 0x7f80d40cc7a0 2023-11-04T05:28:46.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.201+0000 7f80da26f700 1 -- 172.21.15.3:0/1548660110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d40aea20 con 0x7f80d40cc7a0 2023-11-04T05:28:46.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.201+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc012b60 con 0x7f80d40cc7a0 2023-11-04T05:28:46.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.201+0000 7f80da26f700 1 -- 172.21.15.3:0/1548660110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d40aefe0 con 0x7f80d40cc7a0 2023-11-04T05:28:46.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.202+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac0052f0 con 0x7f80d40cc7a0 2023-11-04T05:28:46.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.202+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80cc012cc0 con 0x7f80d40cc7a0 2023-11-04T05:28:46.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.203+0000 7f80d0ff9700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f80b406af00 0x7f80b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.203+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f80cc08bc40 con 0x7f80d40cc7a0 2023-11-04T05:28:46.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.203+0000 7f80d3fff700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f80b406af00 0x7f80b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.203+0000 7f80d3fff700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f80b406af00 0x7f80b406d3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000f60 tx=0x7f80c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:46.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.207+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f80cc05a400 con 0x7f80d40cc7a0 2023-11-04T05:28:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.404+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 --> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80ac000bc0 con 0x7f80b406af00 2023-11-04T05:28:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.413+0000 7f80d0ff9700 1 -- 172.21.15.3:0/1548660110 <== mgr.4099 v2:172.21.15.188:6832/3102759174 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345262 (secure 0 0 0) 0x7f80ac000bc0 con 0x7f80b406af00 2023-11-04T05:28:46.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f80b406af00 msgr2=0x7f80b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f80b406af00 0x7f80b406d3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000f60 tx=0x7f80c4009040 comp rx=0 tx=0).stop 2023-11-04T05:28:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 msgr2=0x7f80d4157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 0x7f80d4157c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f80d40d1640 tx=0x7f80cc004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 shutdown_connections 2023-11-04T05:28:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f80b406af00 0x7f80b406d3c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80d40cc7a0 0x7f80d4157c20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80d406ab20 0x7f80d40b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 --2- 172.21.15.3:0/1548660110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d40728b0 0x7f80d41525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 >> 172.21.15.3:0/1548660110 conn(0x7f80d406df90 msgr2=0x7f80d40b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 shutdown_connections 2023-11-04T05:28:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.417+0000 7f80ca7fc700 1 -- 172.21.15.3:0/1548660110 wait complete. 2023-11-04T05:28:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-11-04T05:28:46.431 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-11-04T05:28:45.756017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2045232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003796"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835820+0000","last_change":"2023-11-04T05:28:21.455216+0000","last_active":"2023-11-04T05:28:25.835820+0000","last_peered":"2023-11-04T05:28:25.835820+0000","last_clean":"2023-11-04T05:28:25.835820+0000","last_became_active":"2023-11-04T05:28:21.453964+0000","last_became_peered":"2023-11-04T05:28:21.453964+0000","last_unstale":"2023-11-04T05:28:25.835820+0000","last_undegraded":"2023-11-04T05:28:25.835820+0000","last_fullsized":"2023-11-04T05:28:25.835820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858340+0000","last_change":"2023-11-04T05:28:21.458543+0000","last_active":"2023-11-04T05:28:25.858340+0000","last_peered":"2023-11-04T05:28:25.858340+0000","last_clean":"2023-11-04T05:28:25.858340+0000","last_became_active":"2023-11-04T05:28:21.458174+0000","last_became_peered":"2023-11-04T05:28:21.458174+0000","last_unstale":"2023-11-04T05:28:25.858340+0000","last_undegraded":"2023-11-04T05:28:25.858340+0000","last_fullsized":"2023-11-04T05:28:25.858340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842597+0000","last_change":"2023-11-04T05:28:21.457190+0000","last_active":"2023-11-04T05:28:25.842597+0000","last_peered":"2023-11-04T05:28:25.842597+0000","last_clean":"2023-11-04T05:28:25.842597+0000","last_became_active":"2023-11-04T05:28:21.456901+0000","last_became_peered":"2023-11-04T05:28:21.456901+0000","last_unstale":"2023-11-04T05:28:25.842597+0000","last_undegraded":"2023-11-04T05:28:25.842597+0000","last_fullsized":"2023-11-04T05:28:25.842597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:00.273535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840758+0000","last_change":"2023-11-04T05:28:21.464308+0000","last_active":"2023-11-04T05:28:25.840758+0000","last_peered":"2023-11-04T05:28:25.840758+0000","last_clean":"2023-11-04T05:28:25.840758+0000","last_became_active":"2023-11-04T05:28:21.464035+0000","last_became_peered":"2023-11-04T05:28:21.464035+0000","last_unstale":"2023-11-04T05:28:25.840758+0000","last_undegraded":"2023-11-04T05:28:25.840758+0000","last_fullsized":"2023-11-04T05:28:25.840758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:21.443272+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:21.442885+0000","last_became_peered":"2023-11-04T05:28:21.442885+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839412+0000","last_change":"2023-11-04T05:28:21.461562+0000","last_active":"2023-11-04T05:28:25.839412+0000","last_peered":"2023-11-04T05:28:25.839412+0000","last_clean":"2023-11-04T05:28:25.839412+0000","last_became_active":"2023-11-04T05:28:21.461276+0000","last_became_peered":"2023-11-04T05:28:21.461276+0000","last_unstale":"2023-11-04T05:28:25.839412+0000","last_undegraded":"2023-11-04T05:28:25.839412+0000","last_fullsized":"2023-11-04T05:28:25.839412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834303+0000","last_change":"2023-11-04T05:28:21.456621+0000","last_active":"2023-11-04T05:28:25.834303+0000","last_peered":"2023-11-04T05:28:25.834303+0000","last_clean":"2023-11-04T05:28:25.834303+0000","last_became_active":"2023-11-04T05:28:21.456356+0000","last_became_peered":"2023-11-04T05:28:21.456356+0000","last_unstale":"2023-11-04T05:28:25.834303+0000","last_undegraded":"2023-11-04T05:28:25.834303+0000","last_fullsized":"2023-11-04T05:28:25.834303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841746+0000","last_change":"2023-11-04T05:28:21.463624+0000","last_active":"2023-11-04T05:28:25.841746+0000","last_peered":"2023-11-04T05:28:25.841746+0000","last_clean":"2023-11-04T05:28:25.841746+0000","last_became_active":"2023-11-04T05:28:21.463336+0000","last_became_peered":"2023-11-04T05:28:21.463336+0000","last_unstale":"2023-11-04T05:28:25.841746+0000","last_undegraded":"2023-11-04T05:28:25.841746+0000","last_fullsized":"2023-11-04T05:28:25.841746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835730+0000","last_change":"2023-11-04T05:28:21.466992+0000","last_active":"2023-11-04T05:28:25.835730+0000","last_peered":"2023-11-04T05:28:25.835730+0000","last_clean":"2023-11-04T05:28:25.835730+0000","last_became_active":"2023-11-04T05:28:21.466638+0000","last_became_peered":"2023-11-04T05:28:21.466638+0000","last_unstale":"2023-11-04T05:28:25.835730+0000","last_undegraded":"2023-11-04T05:28:25.835730+0000","last_fullsized":"2023-11-04T05:28:25.835730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:01:55.625411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835857+0000","last_change":"2023-11-04T05:28:21.466265+0000","last_active":"2023-11-04T05:28:25.835857+0000","last_peered":"2023-11-04T05:28:25.835857+0000","last_clean":"2023-11-04T05:28:25.835857+0000","last_became_active":"2023-11-04T05:28:21.466080+0000","last_became_peered":"2023-11-04T05:28:21.466080+0000","last_unstale":"2023-11-04T05:28:25.835857+0000","last_undegraded":"2023-11-04T05:28:25.835857+0000","last_fullsized":"2023-11-04T05:28:25.835857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835259+0000","last_change":"2023-11-04T05:28:21.465839+0000","last_active":"2023-11-04T05:28:25.835259+0000","last_peered":"2023-11-04T05:28:25.835259+0000","last_clean":"2023-11-04T05:28:25.835259+0000","last_became_active":"2023-11-04T05:28:21.465571+0000","last_became_peered":"2023-11-04T05:28:21.465571+0000","last_unstale":"2023-11-04T05:28:25.835259+0000","last_undegraded":"2023-11-04T05:28:25.835259+0000","last_fullsized":"2023-11-04T05:28:25.835259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843187+0000","last_change":"2023-11-04T05:28:21.459690+0000","last_active":"2023-11-04T05:28:25.843187+0000","last_peered":"2023-11-04T05:28:25.843187+0000","last_clean":"2023-11-04T05:28:25.843187+0000","last_became_active":"2023-11-04T05:28:21.459406+0000","last_became_peered":"2023-11-04T05:28:21.459406+0000","last_unstale":"2023-11-04T05:28:25.843187+0000","last_undegraded":"2023-11-04T05:28:25.843187+0000","last_fullsized":"2023-11-04T05:28:25.843187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840245+0000","last_change":"2023-11-04T05:28:21.452009+0000","last_active":"2023-11-04T05:28:25.840245+0000","last_peered":"2023-11-04T05:28:25.840245+0000","last_clean":"2023-11-04T05:28:25.840245+0000","last_became_active":"2023-11-04T05:28:21.451701+0000","last_became_peered":"2023-11-04T05:28:21.451701+0000","last_unstale":"2023-11-04T05:28:25.840245+0000","last_undegraded":"2023-11-04T05:28:25.840245+0000","last_fullsized":"2023-11-04T05:28:25.840245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842267+0000","last_change":"2023-11-04T05:28:21.449054+0000","last_active":"2023-11-04T05:28:25.842267+0000","last_peered":"2023-11-04T05:28:25.842267+0000","last_clean":"2023-11-04T05:28:25.842267+0000","last_became_active":"2023-11-04T05:28:21.448593+0000","last_became_peered":"2023-11-04T05:28:21.448593+0000","last_unstale":"2023-11-04T05:28:25.842267+0000","last_undegraded":"2023-11-04T05:28:25.842267+0000","last_fullsized":"2023-11-04T05:28:25.842267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:16:04.673589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857682+0000","last_change":"2023-11-04T05:28:21.459859+0000","last_active":"2023-11-04T05:28:25.857682+0000","last_peered":"2023-11-04T05:28:25.857682+0000","last_clean":"2023-11-04T05:28:25.857682+0000","last_became_active":"2023-11-04T05:28:21.459569+0000","last_became_peered":"2023-11-04T05:28:21.459569+0000","last_unstale":"2023-11-04T05:28:25.857682+0000","last_undegraded":"2023-11-04T05:28:25.857682+0000","last_fullsized":"2023-11-04T05:28:25.857682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839549+0000","last_change":"2023-11-04T05:28:21.461640+0000","last_active":"2023-11-04T05:28:25.839549+0000","last_peered":"2023-11-04T05:28:25.839549+0000","last_clean":"2023-11-04T05:28:25.839549+0000","last_became_active":"2023-11-04T05:28:21.461239+0000","last_became_peered":"2023-11-04T05:28:21.461239+0000","last_unstale":"2023-11-04T05:28:25.839549+0000","last_undegraded":"2023-11-04T05:28:25.839549+0000","last_fullsized":"2023-11-04T05:28:25.839549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841199+0000","last_change":"2023-11-04T05:28:21.463999+0000","last_active":"2023-11-04T05:28:25.841199+0000","last_peered":"2023-11-04T05:28:25.841199+0000","last_clean":"2023-11-04T05:28:25.841199+0000","last_became_active":"2023-11-04T05:28:21.463743+0000","last_became_peered":"2023-11-04T05:28:21.463743+0000","last_unstale":"2023-11-04T05:28:25.841199+0000","last_undegraded":"2023-11-04T05:28:25.841199+0000","last_fullsized":"2023-11-04T05:28:25.841199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846908+0000","last_change":"2023-11-04T05:28:21.454396+0000","last_active":"2023-11-04T05:28:25.846908+0000","last_peered":"2023-11-04T05:28:25.846908+0000","last_clean":"2023-11-04T05:28:25.846908+0000","last_became_active":"2023-11-04T05:28:21.454074+0000","last_became_peered":"2023-11-04T05:28:21.454074+0000","last_unstale":"2023-11-04T05:28:25.846908+0000","last_undegraded":"2023-11-04T05:28:25.846908+0000","last_fullsized":"2023-11-04T05:28:25.846908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835087+0000","last_change":"2023-11-04T05:28:21.441379+0000","last_active":"2023-11-04T05:28:25.835087+0000","last_peered":"2023-11-04T05:28:25.835087+0000","last_clean":"2023-11-04T05:28:25.835087+0000","last_became_active":"2023-11-04T05:28:21.441074+0000","last_became_peered":"2023-11-04T05:28:21.441074+0000","last_unstale":"2023-11-04T05:28:25.835087+0000","last_undegraded":"2023-11-04T05:28:25.835087+0000","last_fullsized":"2023-11-04T05:28:25.835087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220068+0000","last_change":"2023-11-04T05:28:25.856956+0000","last_active":"2023-11-04T05:28:28.220068+0000","last_peered":"2023-11-04T05:28:28.220068+0000","last_clean":"2023-11-04T05:28:28.220068+0000","last_became_active":"2023-11-04T05:28:25.856458+0000","last_became_peered":"2023-11-04T05:28:25.856458+0000","last_unstale":"2023-11-04T05:28:28.220068+0000","last_undegraded":"2023-11-04T05:28:28.220068+0000","last_fullsized":"2023-11-04T05:28:28.220068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:01:17.491615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842941+0000","last_change":"2023-11-04T05:28:20.455408+0000","last_active":"2023-11-04T05:28:25.842941+0000","last_peered":"2023-11-04T05:28:25.842941+0000","last_clean":"2023-11-04T05:28:25.842941+0000","last_became_active":"2023-11-04T05:28:20.455106+0000","last_became_peered":"2023-11-04T05:28:20.455106+0000","last_unstale":"2023-11-04T05:28:25.842941+0000","last_undegraded":"2023-11-04T05:28:25.842941+0000","last_fullsized":"2023-11-04T05:28:25.842941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:09:15.898778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840543+0000","last_change":"2023-11-04T05:28:21.452315+0000","last_active":"2023-11-04T05:28:25.840543+0000","last_peered":"2023-11-04T05:28:25.840543+0000","last_clean":"2023-11-04T05:28:25.840543+0000","last_became_active":"2023-11-04T05:28:21.451728+0000","last_became_peered":"2023-11-04T05:28:21.451728+0000","last_unstale":"2023-11-04T05:28:25.840543+0000","last_undegraded":"2023-11-04T05:28:25.840543+0000","last_fullsized":"2023-11-04T05:28:25.840543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742885+0000","last_change":"2023-11-04T05:28:25.861875+0000","last_active":"2023-11-04T05:28:28.742885+0000","last_peered":"2023-11-04T05:28:28.742885+0000","last_clean":"2023-11-04T05:28:28.742885+0000","last_became_active":"2023-11-04T05:28:25.861458+0000","last_became_peered":"2023-11-04T05:28:25.861458+0000","last_unstale":"2023-11-04T05:28:28.742885+0000","last_undegraded":"2023-11-04T05:28:28.742885+0000","last_fullsized":"2023-11-04T05:28:28.742885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.153647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840648+0000","last_change":"2023-11-04T05:28:20.456506+0000","last_active":"2023-11-04T05:28:25.840648+0000","last_peered":"2023-11-04T05:28:25.840648+0000","last_clean":"2023-11-04T05:28:25.840648+0000","last_became_active":"2023-11-04T05:28:20.456086+0000","last_became_peered":"2023-11-04T05:28:20.456086+0000","last_unstale":"2023-11-04T05:28:25.840648+0000","last_undegraded":"2023-11-04T05:28:25.840648+0000","last_fullsized":"2023-11-04T05:28:25.840648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842845+0000","last_change":"2023-11-04T05:28:21.464433+0000","last_active":"2023-11-04T05:28:25.842845+0000","last_peered":"2023-11-04T05:28:25.842845+0000","last_clean":"2023-11-04T05:28:25.842845+0000","last_became_active":"2023-11-04T05:28:21.464133+0000","last_became_peered":"2023-11-04T05:28:21.464133+0000","last_unstale":"2023-11-04T05:28:25.842845+0000","last_undegraded":"2023-11-04T05:28:25.842845+0000","last_fullsized":"2023-11-04T05:28:25.842845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:26:07.829857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743096+0000","last_change":"2023-11-04T05:28:25.858693+0000","last_active":"2023-11-04T05:28:28.743096+0000","last_peered":"2023-11-04T05:28:28.743096+0000","last_clean":"2023-11-04T05:28:28.743096+0000","last_became_active":"2023-11-04T05:28:25.858151+0000","last_became_peered":"2023-11-04T05:28:25.858151+0000","last_unstale":"2023-11-04T05:28:28.743096+0000","last_undegraded":"2023-11-04T05:28:28.743096+0000","last_fullsized":"2023-11-04T05:28:28.743096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:33:57.728742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:20.448994+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:20.448722+0000","last_became_peered":"2023-11-04T05:28:20.448722+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839002+0000","last_change":"2023-11-04T05:28:21.464488+0000","last_active":"2023-11-04T05:28:25.839002+0000","last_peered":"2023-11-04T05:28:25.839002+0000","last_clean":"2023-11-04T05:28:25.839002+0000","last_became_active":"2023-11-04T05:28:21.462679+0000","last_became_peered":"2023-11-04T05:28:21.462679+0000","last_unstale":"2023-11-04T05:28:25.839002+0000","last_undegraded":"2023-11-04T05:28:25.839002+0000","last_fullsized":"2023-11-04T05:28:25.839002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743030+0000","last_change":"2023-11-04T05:28:25.854516+0000","last_active":"2023-11-04T05:28:28.743030+0000","last_peered":"2023-11-04T05:28:28.743030+0000","last_clean":"2023-11-04T05:28:28.743030+0000","last_became_active":"2023-11-04T05:28:25.854162+0000","last_became_peered":"2023-11-04T05:28:25.854162+0000","last_unstale":"2023-11-04T05:28:28.743030+0000","last_undegraded":"2023-11-04T05:28:28.743030+0000","last_fullsized":"2023-11-04T05:28:28.743030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:06.645428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834998+0000","last_change":"2023-11-04T05:28:20.452770+0000","last_active":"2023-11-04T05:28:25.834998+0000","last_peered":"2023-11-04T05:28:25.834998+0000","last_clean":"2023-11-04T05:28:25.834998+0000","last_became_active":"2023-11-04T05:28:20.452404+0000","last_became_peered":"2023-11-04T05:28:20.452404+0000","last_unstale":"2023-11-04T05:28:25.834998+0000","last_undegraded":"2023-11-04T05:28:25.834998+0000","last_fullsized":"2023-11-04T05:28:25.834998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842144+0000","last_change":"2023-11-04T05:28:21.463294+0000","last_active":"2023-11-04T05:28:25.842144+0000","last_peered":"2023-11-04T05:28:25.842144+0000","last_clean":"2023-11-04T05:28:25.842144+0000","last_became_active":"2023-11-04T05:28:21.463028+0000","last_became_peered":"2023-11-04T05:28:21.463028+0000","last_unstale":"2023-11-04T05:28:25.842144+0000","last_undegraded":"2023-11-04T05:28:25.842144+0000","last_fullsized":"2023-11-04T05:28:25.842144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220306+0000","last_change":"2023-11-04T05:28:25.864434+0000","last_active":"2023-11-04T05:28:28.220306+0000","last_peered":"2023-11-04T05:28:28.220306+0000","last_clean":"2023-11-04T05:28:28.220306+0000","last_became_active":"2023-11-04T05:28:25.863978+0000","last_became_peered":"2023-11-04T05:28:25.863978+0000","last_unstale":"2023-11-04T05:28:28.220306+0000","last_undegraded":"2023-11-04T05:28:28.220306+0000","last_fullsized":"2023-11-04T05:28:28.220306+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:22:02.298285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840436+0000","last_change":"2023-11-04T05:28:20.451845+0000","last_active":"2023-11-04T05:28:25.840436+0000","last_peered":"2023-11-04T05:28:25.840436+0000","last_clean":"2023-11-04T05:28:25.840436+0000","last_became_active":"2023-11-04T05:28:20.451350+0000","last_became_peered":"2023-11-04T05:28:20.451350+0000","last_unstale":"2023-11-04T05:28:25.840436+0000","last_undegraded":"2023-11-04T05:28:25.840436+0000","last_fullsized":"2023-11-04T05:28:25.840436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.837664+0000","last_change":"2023-11-04T05:28:21.454758+0000","last_active":"2023-11-04T05:28:25.837664+0000","last_peered":"2023-11-04T05:28:25.837664+0000","last_clean":"2023-11-04T05:28:25.837664+0000","last_became_active":"2023-11-04T05:28:21.449696+0000","last_became_peered":"2023-11-04T05:28:21.449696+0000","last_unstale":"2023-11-04T05:28:25.837664+0000","last_undegraded":"2023-11-04T05:28:25.837664+0000","last_fullsized":"2023-11-04T05:28:25.837664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:13:54.163937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.177060+0000","last_change":"2023-11-04T05:28:25.869885+0000","last_active":"2023-11-04T05:28:28.177060+0000","last_peered":"2023-11-04T05:28:28.177060+0000","last_clean":"2023-11-04T05:28:28.177060+0000","last_became_active":"2023-11-04T05:28:25.869295+0000","last_became_peered":"2023-11-04T05:28:25.869295+0000","last_unstale":"2023-11-04T05:28:28.177060+0000","last_undegraded":"2023-11-04T05:28:28.177060+0000","last_fullsized":"2023-11-04T05:28:28.177060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:02:41.884046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834648+0000","last_change":"2023-11-04T05:28:20.445804+0000","last_active":"2023-11-04T05:28:25.834648+0000","last_peered":"2023-11-04T05:28:25.834648+0000","last_clean":"2023-11-04T05:28:25.834648+0000","last_became_active":"2023-11-04T05:28:20.445476+0000","last_became_peered":"2023-11-04T05:28:20.445476+0000","last_unstale":"2023-11-04T05:28:25.834648+0000","last_undegraded":"2023-11-04T05:28:25.834648+0000","last_fullsized":"2023-11-04T05:28:25.834648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:22:03.756669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842664+0000","last_change":"2023-11-04T05:28:21.460996+0000","last_active":"2023-11-04T05:28:25.842664+0000","last_peered":"2023-11-04T05:28:25.842664+0000","last_clean":"2023-11-04T05:28:25.842664+0000","last_became_active":"2023-11-04T05:28:21.460451+0000","last_became_peered":"2023-11-04T05:28:21.460451+0000","last_unstale":"2023-11-04T05:28:25.842664+0000","last_undegraded":"2023-11-04T05:28:25.842664+0000","last_fullsized":"2023-11-04T05:28:25.842664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:02.249647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838965+0000","last_change":"2023-11-04T05:28:25.863990+0000","last_active":"2023-11-04T05:28:26.838965+0000","last_peered":"2023-11-04T05:28:26.838965+0000","last_clean":"2023-11-04T05:28:26.838965+0000","last_became_active":"2023-11-04T05:28:25.863587+0000","last_became_peered":"2023-11-04T05:28:25.863587+0000","last_unstale":"2023-11-04T05:28:26.838965+0000","last_undegraded":"2023-11-04T05:28:26.838965+0000","last_fullsized":"2023-11-04T05:28:26.838965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841913+0000","last_change":"2023-11-04T05:28:20.448359+0000","last_active":"2023-11-04T05:28:25.841913+0000","last_peered":"2023-11-04T05:28:25.841913+0000","last_clean":"2023-11-04T05:28:25.841913+0000","last_became_active":"2023-11-04T05:28:20.448037+0000","last_became_peered":"2023-11-04T05:28:20.448037+0000","last_unstale":"2023-11-04T05:28:25.841913+0000","last_undegraded":"2023-11-04T05:28:25.841913+0000","last_fullsized":"2023-11-04T05:28:25.841913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:02:08.721929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839909+0000","last_change":"2023-11-04T05:28:21.459362+0000","last_active":"2023-11-04T05:28:25.839909+0000","last_peered":"2023-11-04T05:28:25.839909+0000","last_clean":"2023-11-04T05:28:25.839909+0000","last_became_active":"2023-11-04T05:28:21.458878+0000","last_became_peered":"2023-11-04T05:28:21.458878+0000","last_unstale":"2023-11-04T05:28:25.839909+0000","last_undegraded":"2023-11-04T05:28:25.839909+0000","last_fullsized":"2023-11-04T05:28:25.839909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919977+0000","last_change":"2023-11-04T05:28:25.865090+0000","last_active":"2023-11-04T05:28:27.919977+0000","last_peered":"2023-11-04T05:28:27.919977+0000","last_clean":"2023-11-04T05:28:27.919977+0000","last_became_active":"2023-11-04T05:28:25.864492+0000","last_became_peered":"2023-11-04T05:28:25.864492+0000","last_unstale":"2023-11-04T05:28:27.919977+0000","last_undegraded":"2023-11-04T05:28:27.919977+0000","last_fullsized":"2023-11-04T05:28:27.919977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:52:12.074596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839654+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.839654+0000","last_peered":"2023-11-04T05:28:25.839654+0000","last_clean":"2023-11-04T05:28:25.839654+0000","last_became_active":"2023-11-04T05:28:20.451322+0000","last_became_peered":"2023-11-04T05:28:20.451322+0000","last_unstale":"2023-11-04T05:28:25.839654+0000","last_undegraded":"2023-11-04T05:28:25.839654+0000","last_fullsized":"2023-11-04T05:28:25.839654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:16.770342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857938+0000","last_change":"2023-11-04T05:28:21.446085+0000","last_active":"2023-11-04T05:28:25.857938+0000","last_peered":"2023-11-04T05:28:25.857938+0000","last_clean":"2023-11-04T05:28:25.857938+0000","last_became_active":"2023-11-04T05:28:21.445831+0000","last_became_peered":"2023-11-04T05:28:21.445831+0000","last_unstale":"2023-11-04T05:28:25.857938+0000","last_undegraded":"2023-11-04T05:28:25.857938+0000","last_fullsized":"2023-11-04T05:28:25.857938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219926+0000","last_change":"2023-11-04T05:28:25.853036+0000","last_active":"2023-11-04T05:28:28.219926+0000","last_peered":"2023-11-04T05:28:28.219926+0000","last_clean":"2023-11-04T05:28:28.219926+0000","last_became_active":"2023-11-04T05:28:25.852217+0000","last_became_peered":"2023-11-04T05:28:25.852217+0000","last_unstale":"2023-11-04T05:28:28.219926+0000","last_undegraded":"2023-11-04T05:28:28.219926+0000","last_fullsized":"2023-11-04T05:28:28.219926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835077+0000","last_change":"2023-11-04T05:28:20.448588+0000","last_active":"2023-11-04T05:28:25.835077+0000","last_peered":"2023-11-04T05:28:25.835077+0000","last_clean":"2023-11-04T05:28:25.835077+0000","last_became_active":"2023-11-04T05:28:20.448330+0000","last_became_peered":"2023-11-04T05:28:20.448330+0000","last_unstale":"2023-11-04T05:28:25.835077+0000","last_undegraded":"2023-11-04T05:28:25.835077+0000","last_fullsized":"2023-11-04T05:28:25.835077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841018+0000","last_change":"2023-11-04T05:28:21.462557+0000","last_active":"2023-11-04T05:28:25.841018+0000","last_peered":"2023-11-04T05:28:25.841018+0000","last_clean":"2023-11-04T05:28:25.841018+0000","last_became_active":"2023-11-04T05:28:21.462247+0000","last_became_peered":"2023-11-04T05:28:21.462247+0000","last_unstale":"2023-11-04T05:28:25.841018+0000","last_undegraded":"2023-11-04T05:28:25.841018+0000","last_fullsized":"2023-11-04T05:28:25.841018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219973+0000","last_change":"2023-11-04T05:28:25.860094+0000","last_active":"2023-11-04T05:28:28.219973+0000","last_peered":"2023-11-04T05:28:28.219973+0000","last_clean":"2023-11-04T05:28:28.219973+0000","last_became_active":"2023-11-04T05:28:25.859164+0000","last_became_peered":"2023-11-04T05:28:25.859164+0000","last_unstale":"2023-11-04T05:28:28.219973+0000","last_undegraded":"2023-11-04T05:28:28.219973+0000","last_fullsized":"2023-11-04T05:28:28.219973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840932+0000","last_change":"2023-11-04T05:28:20.451529+0000","last_active":"2023-11-04T05:28:25.840932+0000","last_peered":"2023-11-04T05:28:25.840932+0000","last_clean":"2023-11-04T05:28:25.840932+0000","last_became_active":"2023-11-04T05:28:20.451236+0000","last_became_peered":"2023-11-04T05:28:20.451236+0000","last_unstale":"2023-11-04T05:28:25.840932+0000","last_undegraded":"2023-11-04T05:28:25.840932+0000","last_fullsized":"2023-11-04T05:28:25.840932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843021+0000","last_change":"2023-11-04T05:28:21.464721+0000","last_active":"2023-11-04T05:28:25.843021+0000","last_peered":"2023-11-04T05:28:25.843021+0000","last_clean":"2023-11-04T05:28:25.843021+0000","last_became_active":"2023-11-04T05:28:21.464403+0000","last_became_peered":"2023-11-04T05:28:21.464403+0000","last_unstale":"2023-11-04T05:28:25.843021+0000","last_undegraded":"2023-11-04T05:28:25.843021+0000","last_fullsized":"2023-11-04T05:28:25.843021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:40.757866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743335+0000","last_change":"2023-11-04T05:28:25.858588+0000","last_active":"2023-11-04T05:28:28.743335+0000","last_peered":"2023-11-04T05:28:28.743335+0000","last_clean":"2023-11-04T05:28:28.743335+0000","last_became_active":"2023-11-04T05:28:25.858202+0000","last_became_peered":"2023-11-04T05:28:25.858202+0000","last_unstale":"2023-11-04T05:28:28.743335+0000","last_undegraded":"2023-11-04T05:28:28.743335+0000","last_fullsized":"2023-11-04T05:28:28.743335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:36:16.890049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858629+0000","last_change":"2023-11-04T05:28:20.451067+0000","last_active":"2023-11-04T05:28:25.858629+0000","last_peered":"2023-11-04T05:28:25.858629+0000","last_clean":"2023-11-04T05:28:25.858629+0000","last_became_active":"2023-11-04T05:28:20.450732+0000","last_became_peered":"2023-11-04T05:28:20.450732+0000","last_unstale":"2023-11-04T05:28:25.858629+0000","last_undegraded":"2023-11-04T05:28:25.858629+0000","last_fullsized":"2023-11-04T05:28:25.858629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843093+0000","last_change":"2023-11-04T05:28:21.457579+0000","last_active":"2023-11-04T05:28:25.843093+0000","last_peered":"2023-11-04T05:28:25.843093+0000","last_clean":"2023-11-04T05:28:25.843093+0000","last_became_active":"2023-11-04T05:28:21.457314+0000","last_became_peered":"2023-11-04T05:28:21.457314+0000","last_unstale":"2023-11-04T05:28:25.843093+0000","last_undegraded":"2023-11-04T05:28:25.843093+0000","last_fullsized":"2023-11-04T05:28:25.843093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832422+0000","last_change":"2023-11-04T05:28:25.863159+0000","last_active":"2023-11-04T05:28:28.832422+0000","last_peered":"2023-11-04T05:28:28.832422+0000","last_clean":"2023-11-04T05:28:28.832422+0000","last_became_active":"2023-11-04T05:28:25.862645+0000","last_became_peered":"2023-11-04T05:28:25.862645+0000","last_unstale":"2023-11-04T05:28:28.832422+0000","last_undegraded":"2023-11-04T05:28:28.832422+0000","last_fullsized":"2023-11-04T05:28:28.832422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:41.867932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:31.743983+0000","last_change":"2023-11-04T05:28:20.449394+0000","last_active":"2023-11-04T05:28:31.743983+0000","last_peered":"2023-11-04T05:28:31.743983+0000","last_clean":"2023-11-04T05:28:31.743983+0000","last_became_active":"2023-11-04T05:28:20.449060+0000","last_became_peered":"2023-11-04T05:28:20.449060+0000","last_unstale":"2023-11-04T05:28:31.743983+0000","last_undegraded":"2023-11-04T05:28:31.743983+0000","last_fullsized":"2023-11-04T05:28:31.743983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835007+0000","last_change":"2023-11-04T05:28:21.459298+0000","last_active":"2023-11-04T05:28:25.835007+0000","last_peered":"2023-11-04T05:28:25.835007+0000","last_clean":"2023-11-04T05:28:25.835007+0000","last_became_active":"2023-11-04T05:28:21.459046+0000","last_became_peered":"2023-11-04T05:28:21.459046+0000","last_unstale":"2023-11-04T05:28:25.835007+0000","last_undegraded":"2023-11-04T05:28:25.835007+0000","last_fullsized":"2023-11-04T05:28:25.835007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836097+0000","last_change":"2023-11-04T05:28:25.853324+0000","last_active":"2023-11-04T05:28:26.836097+0000","last_peered":"2023-11-04T05:28:26.836097+0000","last_clean":"2023-11-04T05:28:26.836097+0000","last_became_active":"2023-11-04T05:28:25.852913+0000","last_became_peered":"2023-11-04T05:28:25.852913+0000","last_unstale":"2023-11-04T05:28:26.836097+0000","last_undegraded":"2023-11-04T05:28:26.836097+0000","last_fullsized":"2023-11-04T05:28:26.836097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842933+0000","last_change":"2023-11-04T05:28:20.451652+0000","last_active":"2023-11-04T05:28:25.842933+0000","last_peered":"2023-11-04T05:28:25.842933+0000","last_clean":"2023-11-04T05:28:25.842933+0000","last_became_active":"2023-11-04T05:28:20.451415+0000","last_became_peered":"2023-11-04T05:28:20.451415+0000","last_unstale":"2023-11-04T05:28:25.842933+0000","last_undegraded":"2023-11-04T05:28:25.842933+0000","last_fullsized":"2023-11-04T05:28:25.842933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.132028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842574+0000","last_change":"2023-11-04T05:28:21.454210+0000","last_active":"2023-11-04T05:28:25.842574+0000","last_peered":"2023-11-04T05:28:25.842574+0000","last_clean":"2023-11-04T05:28:25.842574+0000","last_became_active":"2023-11-04T05:28:21.453894+0000","last_became_peered":"2023-11-04T05:28:21.453894+0000","last_unstale":"2023-11-04T05:28:25.842574+0000","last_undegraded":"2023-11-04T05:28:25.842574+0000","last_fullsized":"2023-11-04T05:28:25.842574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:28:59.983010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220347+0000","last_change":"2023-11-04T05:28:25.864286+0000","last_active":"2023-11-04T05:28:28.220347+0000","last_peered":"2023-11-04T05:28:28.220347+0000","last_clean":"2023-11-04T05:28:28.220347+0000","last_became_active":"2023-11-04T05:28:25.863831+0000","last_became_peered":"2023-11-04T05:28:25.863831+0000","last_unstale":"2023-11-04T05:28:28.220347+0000","last_undegraded":"2023-11-04T05:28:28.220347+0000","last_fullsized":"2023-11-04T05:28:28.220347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:40:57.121972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839219+0000","last_change":"2023-11-04T05:28:20.456235+0000","last_active":"2023-11-04T05:28:25.839219+0000","last_peered":"2023-11-04T05:28:25.839219+0000","last_clean":"2023-11-04T05:28:25.839219+0000","last_became_active":"2023-11-04T05:28:20.455992+0000","last_became_peered":"2023-11-04T05:28:20.455992+0000","last_unstale":"2023-11-04T05:28:25.839219+0000","last_undegraded":"2023-11-04T05:28:25.839219+0000","last_fullsized":"2023-11-04T05:28:25.839219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834802+0000","last_change":"2023-11-04T05:28:21.460351+0000","last_active":"2023-11-04T05:28:25.834802+0000","last_peered":"2023-11-04T05:28:25.834802+0000","last_clean":"2023-11-04T05:28:25.834802+0000","last_became_active":"2023-11-04T05:28:21.459737+0000","last_became_peered":"2023-11-04T05:28:21.459737+0000","last_unstale":"2023-11-04T05:28:25.834802+0000","last_undegraded":"2023-11-04T05:28:25.834802+0000","last_fullsized":"2023-11-04T05:28:25.834802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220525+0000","last_change":"2023-11-04T05:28:25.868351+0000","last_active":"2023-11-04T05:28:28.220525+0000","last_peered":"2023-11-04T05:28:28.220525+0000","last_clean":"2023-11-04T05:28:28.220525+0000","last_became_active":"2023-11-04T05:28:25.867909+0000","last_became_peered":"2023-11-04T05:28:25.867909+0000","last_unstale":"2023-11-04T05:28:28.220525+0000","last_undegraded":"2023-11-04T05:28:28.220525+0000","last_fullsized":"2023-11-04T05:28:28.220525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:08:06.813956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833857+0000","last_change":"2023-11-04T05:28:20.447033+0000","last_active":"2023-11-04T05:28:25.833857+0000","last_peered":"2023-11-04T05:28:25.833857+0000","last_clean":"2023-11-04T05:28:25.833857+0000","last_became_active":"2023-11-04T05:28:20.446767+0000","last_became_peered":"2023-11-04T05:28:20.446767+0000","last_unstale":"2023-11-04T05:28:25.833857+0000","last_undegraded":"2023-11-04T05:28:25.833857+0000","last_fullsized":"2023-11-04T05:28:25.833857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836084+0000","last_change":"2023-11-04T05:28:21.457261+0000","last_active":"2023-11-04T05:28:25.836084+0000","last_peered":"2023-11-04T05:28:25.836084+0000","last_clean":"2023-11-04T05:28:25.836084+0000","last_became_active":"2023-11-04T05:28:21.456982+0000","last_became_peered":"2023-11-04T05:28:21.456982+0000","last_unstale":"2023-11-04T05:28:25.836084+0000","last_undegraded":"2023-11-04T05:28:25.836084+0000","last_fullsized":"2023-11-04T05:28:25.836084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919838+0000","last_change":"2023-11-04T05:28:25.854585+0000","last_active":"2023-11-04T05:28:27.919838+0000","last_peered":"2023-11-04T05:28:27.919838+0000","last_clean":"2023-11-04T05:28:27.919838+0000","last_became_active":"2023-11-04T05:28:25.854151+0000","last_became_peered":"2023-11-04T05:28:25.854151+0000","last_unstale":"2023-11-04T05:28:27.919838+0000","last_undegraded":"2023-11-04T05:28:27.919838+0000","last_fullsized":"2023-11-04T05:28:27.919838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:05:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.844868+0000","last_change":"2023-11-04T05:28:20.448118+0000","last_active":"2023-11-04T05:28:25.844868+0000","last_peered":"2023-11-04T05:28:25.844868+0000","last_clean":"2023-11-04T05:28:25.844868+0000","last_became_active":"2023-11-04T05:28:20.447780+0000","last_became_peered":"2023-11-04T05:28:20.447780+0000","last_unstale":"2023-11-04T05:28:25.844868+0000","last_undegraded":"2023-11-04T05:28:25.844868+0000","last_fullsized":"2023-11-04T05:28:25.844868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:45:52.143624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834466+0000","last_change":"2023-11-04T05:28:21.458858+0000","last_active":"2023-11-04T05:28:25.834466+0000","last_peered":"2023-11-04T05:28:25.834466+0000","last_clean":"2023-11-04T05:28:25.834466+0000","last_became_active":"2023-11-04T05:28:21.458577+0000","last_became_peered":"2023-11-04T05:28:21.458577+0000","last_unstale":"2023-11-04T05:28:25.834466+0000","last_undegraded":"2023-11-04T05:28:25.834466+0000","last_fullsized":"2023-11-04T05:28:25.834466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742613+0000","last_change":"2023-11-04T05:28:25.865986+0000","last_active":"2023-11-04T05:28:28.742613+0000","last_peered":"2023-11-04T05:28:28.742613+0000","last_clean":"2023-11-04T05:28:28.742613+0000","last_became_active":"2023-11-04T05:28:25.865476+0000","last_became_peered":"2023-11-04T05:28:25.865476+0000","last_unstale":"2023-11-04T05:28:28.742613+0000","last_undegraded":"2023-11-04T05:28:28.742613+0000","last_fullsized":"2023-11-04T05:28:28.742613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:23.545163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842973+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.842973+0000","last_peered":"2023-11-04T05:28:25.842973+0000","last_clean":"2023-11-04T05:28:25.842973+0000","last_became_active":"2023-11-04T05:28:20.451353+0000","last_became_peered":"2023-11-04T05:28:20.451353+0000","last_unstale":"2023-11-04T05:28:25.842973+0000","last_undegraded":"2023-11-04T05:28:25.842973+0000","last_fullsized":"2023-11-04T05:28:25.842973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:11:52.224724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835325+0000","last_change":"2023-11-04T05:28:21.460107+0000","last_active":"2023-11-04T05:28:25.835325+0000","last_peered":"2023-11-04T05:28:25.835325+0000","last_clean":"2023-11-04T05:28:25.835325+0000","last_became_active":"2023-11-04T05:28:21.458229+0000","last_became_peered":"2023-11-04T05:28:21.458229+0000","last_unstale":"2023-11-04T05:28:25.835325+0000","last_undegraded":"2023-11-04T05:28:25.835325+0000","last_fullsized":"2023-11-04T05:28:25.835325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832264+0000","last_change":"2023-11-04T05:28:25.863802+0000","last_active":"2023-11-04T05:28:28.832264+0000","last_peered":"2023-11-04T05:28:28.832264+0000","last_clean":"2023-11-04T05:28:28.832264+0000","last_became_active":"2023-11-04T05:28:25.863268+0000","last_became_peered":"2023-11-04T05:28:25.863268+0000","last_unstale":"2023-11-04T05:28:28.832264+0000","last_undegraded":"2023-11-04T05:28:28.832264+0000","last_fullsized":"2023-11-04T05:28:28.832264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:30.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862531+0000","last_change":"2023-11-04T05:28:20.453776+0000","last_active":"2023-11-04T05:28:25.862531+0000","last_peered":"2023-11-04T05:28:25.862531+0000","last_clean":"2023-11-04T05:28:25.862531+0000","last_became_active":"2023-11-04T05:28:20.453564+0000","last_became_peered":"2023-11-04T05:28:20.453564+0000","last_unstale":"2023-11-04T05:28:25.862531+0000","last_undegraded":"2023-11-04T05:28:25.862531+0000","last_fullsized":"2023-11-04T05:28:25.862531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842239+0000","last_change":"2023-11-04T05:28:21.463249+0000","last_active":"2023-11-04T05:28:25.842239+0000","last_peered":"2023-11-04T05:28:25.842239+0000","last_clean":"2023-11-04T05:28:25.842239+0000","last_became_active":"2023-11-04T05:28:21.462964+0000","last_became_peered":"2023-11-04T05:28:21.462964+0000","last_unstale":"2023-11-04T05:28:25.842239+0000","last_undegraded":"2023-11-04T05:28:25.842239+0000","last_fullsized":"2023-11-04T05:28:25.842239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832100+0000","last_change":"2023-11-04T05:28:25.866264+0000","last_active":"2023-11-04T05:28:28.832100+0000","last_peered":"2023-11-04T05:28:28.832100+0000","last_clean":"2023-11-04T05:28:28.832100+0000","last_became_active":"2023-11-04T05:28:25.865946+0000","last_became_peered":"2023-11-04T05:28:25.865946+0000","last_unstale":"2023-11-04T05:28:28.832100+0000","last_undegraded":"2023-11-04T05:28:28.832100+0000","last_fullsized":"2023-11-04T05:28:28.832100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839118+0000","last_change":"2023-11-04T05:28:20.449195+0000","last_active":"2023-11-04T05:28:25.839118+0000","last_peered":"2023-11-04T05:28:25.839118+0000","last_clean":"2023-11-04T05:28:25.839118+0000","last_became_active":"2023-11-04T05:28:20.448862+0000","last_became_peered":"2023-11-04T05:28:20.448862+0000","last_unstale":"2023-11-04T05:28:25.839118+0000","last_undegraded":"2023-11-04T05:28:25.839118+0000","last_fullsized":"2023-11-04T05:28:25.839118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:32.638827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843378+0000","last_change":"2023-11-04T05:28:21.464603+0000","last_active":"2023-11-04T05:28:25.843378+0000","last_peered":"2023-11-04T05:28:25.843378+0000","last_clean":"2023-11-04T05:28:25.843378+0000","last_became_active":"2023-11-04T05:28:21.464337+0000","last_became_peered":"2023-11-04T05:28:21.464337+0000","last_unstale":"2023-11-04T05:28:25.843378+0000","last_undegraded":"2023-11-04T05:28:25.843378+0000","last_fullsized":"2023-11-04T05:28:25.843378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:44.443386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832325+0000","last_change":"2023-11-04T05:28:25.864109+0000","last_active":"2023-11-04T05:28:28.832325+0000","last_peered":"2023-11-04T05:28:28.832325+0000","last_clean":"2023-11-04T05:28:28.832325+0000","last_became_active":"2023-11-04T05:28:25.863293+0000","last_became_peered":"2023-11-04T05:28:25.863293+0000","last_unstale":"2023-11-04T05:28:28.832325+0000","last_undegraded":"2023-11-04T05:28:28.832325+0000","last_fullsized":"2023-11-04T05:28:28.832325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:13.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857963+0000","last_change":"2023-11-04T05:28:21.448655+0000","last_active":"2023-11-04T05:28:25.857963+0000","last_peered":"2023-11-04T05:28:25.857963+0000","last_clean":"2023-11-04T05:28:25.857963+0000","last_became_active":"2023-11-04T05:28:21.448355+0000","last_became_peered":"2023-11-04T05:28:21.448355+0000","last_unstale":"2023-11-04T05:28:25.857963+0000","last_undegraded":"2023-11-04T05:28:25.857963+0000","last_fullsized":"2023-11-04T05:28:25.857963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845533+0000","last_change":"2023-11-04T05:28:20.453706+0000","last_active":"2023-11-04T05:28:25.845533+0000","last_peered":"2023-11-04T05:28:25.845533+0000","last_clean":"2023-11-04T05:28:25.845533+0000","last_became_active":"2023-11-04T05:28:20.453428+0000","last_became_peered":"2023-11-04T05:28:20.453428+0000","last_unstale":"2023-11-04T05:28:25.845533+0000","last_undegraded":"2023-11-04T05:28:25.845533+0000","last_fullsized":"2023-11-04T05:28:25.845533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:43.438258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838700+0000","last_change":"2023-11-04T05:28:25.852681+0000","last_active":"2023-11-04T05:28:26.838700+0000","last_peered":"2023-11-04T05:28:26.838700+0000","last_clean":"2023-11-04T05:28:26.838700+0000","last_became_active":"2023-11-04T05:28:25.852276+0000","last_became_peered":"2023-11-04T05:28:25.852276+0000","last_unstale":"2023-11-04T05:28:26.838700+0000","last_undegraded":"2023-11-04T05:28:26.838700+0000","last_fullsized":"2023-11-04T05:28:26.838700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:28.005917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840730+0000","last_change":"2023-11-04T05:28:21.466480+0000","last_active":"2023-11-04T05:28:25.840730+0000","last_peered":"2023-11-04T05:28:25.840730+0000","last_clean":"2023-11-04T05:28:25.840730+0000","last_became_active":"2023-11-04T05:28:21.466065+0000","last_became_peered":"2023-11-04T05:28:21.466065+0000","last_unstale":"2023-11-04T05:28:25.840730+0000","last_undegraded":"2023-11-04T05:28:25.840730+0000","last_fullsized":"2023-11-04T05:28:25.840730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.838807+0000","last_change":"2023-11-04T05:28:20.451336+0000","last_active":"2023-11-04T05:28:25.838807+0000","last_peered":"2023-11-04T05:28:25.838807+0000","last_clean":"2023-11-04T05:28:25.838807+0000","last_became_active":"2023-11-04T05:28:20.451000+0000","last_became_peered":"2023-11-04T05:28:20.451000+0000","last_unstale":"2023-11-04T05:28:25.838807+0000","last_undegraded":"2023-11-04T05:28:25.838807+0000","last_fullsized":"2023-11-04T05:28:25.838807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:46.798012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834100+0000","last_change":"2023-11-04T05:28:11.829985+0000","last_active":"2023-11-04T05:28:25.834100+0000","last_peered":"2023-11-04T05:28:25.834100+0000","last_clean":"2023-11-04T05:28:25.834100+0000","last_became_active":"2023-11-04T05:28:11.829705+0000","last_became_peered":"2023-11-04T05:28:11.829705+0000","last_unstale":"2023-11-04T05:28:25.834100+0000","last_undegraded":"2023-11-04T05:28:25.834100+0000","last_fullsized":"2023-11-04T05:28:25.834100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_clean_scrub_stamp":"2023-11-04T05:28:10.378947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:46:15.100605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919902+0000","last_change":"2023-11-04T05:28:25.865855+0000","last_active":"2023-11-04T05:28:27.919902+0000","last_peered":"2023-11-04T05:28:27.919902+0000","last_clean":"2023-11-04T05:28:27.919902+0000","last_became_active":"2023-11-04T05:28:25.865538+0000","last_became_peered":"2023-11-04T05:28:25.865538+0000","last_unstale":"2023-11-04T05:28:27.919902+0000","last_undegraded":"2023-11-04T05:28:27.919902+0000","last_fullsized":"2023-11-04T05:28:27.919902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:24:58.810725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842382+0000","last_change":"2023-11-04T05:28:21.461543+0000","last_active":"2023-11-04T05:28:25.842382+0000","last_peered":"2023-11-04T05:28:25.842382+0000","last_clean":"2023-11-04T05:28:25.842382+0000","last_became_active":"2023-11-04T05:28:21.461195+0000","last_became_peered":"2023-11-04T05:28:21.461195+0000","last_unstale":"2023-11-04T05:28:25.842382+0000","last_undegraded":"2023-11-04T05:28:25.842382+0000","last_fullsized":"2023-11-04T05:28:25.842382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:28:07.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834194+0000","last_change":"2023-11-04T05:28:20.450213+0000","last_active":"2023-11-04T05:28:25.834194+0000","last_peered":"2023-11-04T05:28:25.834194+0000","last_clean":"2023-11-04T05:28:25.834194+0000","last_became_active":"2023-11-04T05:28:20.449957+0000","last_became_peered":"2023-11-04T05:28:20.449957+0000","last_unstale":"2023-11-04T05:28:25.834194+0000","last_undegraded":"2023-11-04T05:28:25.834194+0000","last_fullsized":"2023-11-04T05:28:25.834194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:30:12.799059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919653+0000","last_change":"2023-11-04T05:28:25.861935+0000","last_active":"2023-11-04T05:28:27.919653+0000","last_peered":"2023-11-04T05:28:27.919653+0000","last_clean":"2023-11-04T05:28:27.919653+0000","last_became_active":"2023-11-04T05:28:25.861471+0000","last_became_peered":"2023-11-04T05:28:25.861471+0000","last_unstale":"2023-11-04T05:28:27.919653+0000","last_undegraded":"2023-11-04T05:28:27.919653+0000","last_fullsized":"2023-11-04T05:28:27.919653+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:53:47.409110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834605+0000","last_change":"2023-11-04T05:28:20.455821+0000","last_active":"2023-11-04T05:28:25.834605+0000","last_peered":"2023-11-04T05:28:25.834605+0000","last_clean":"2023-11-04T05:28:25.834605+0000","last_became_active":"2023-11-04T05:28:20.455538+0000","last_became_peered":"2023-11-04T05:28:20.455538+0000","last_unstale":"2023-11-04T05:28:25.834605+0000","last_undegraded":"2023-11-04T05:28:25.834605+0000","last_fullsized":"2023-11-04T05:28:25.834605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862653+0000","last_change":"2023-11-04T05:28:21.446022+0000","last_active":"2023-11-04T05:28:25.862653+0000","last_peered":"2023-11-04T05:28:25.862653+0000","last_clean":"2023-11-04T05:28:25.862653+0000","last_became_active":"2023-11-04T05:28:21.445727+0000","last_became_peered":"2023-11-04T05:28:21.445727+0000","last_unstale":"2023-11-04T05:28:25.862653+0000","last_undegraded":"2023-11-04T05:28:25.862653+0000","last_fullsized":"2023-11-04T05:28:25.862653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219703+0000","last_change":"2023-11-04T05:28:25.860350+0000","last_active":"2023-11-04T05:28:28.219703+0000","last_peered":"2023-11-04T05:28:28.219703+0000","last_clean":"2023-11-04T05:28:28.219703+0000","last_became_active":"2023-11-04T05:28:25.859724+0000","last_became_peered":"2023-11-04T05:28:25.859724+0000","last_unstale":"2023-11-04T05:28:28.219703+0000","last_undegraded":"2023-11-04T05:28:28.219703+0000","last_fullsized":"2023-11-04T05:28:28.219703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:33.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.356335+0000","last_change":"2023-11-04T05:28:20.449071+0000","last_active":"2023-11-04T05:28:26.356335+0000","last_peered":"2023-11-04T05:28:26.356335+0000","last_clean":"2023-11-04T05:28:26.356335+0000","last_became_active":"2023-11-04T05:28:20.448778+0000","last_became_peered":"2023-11-04T05:28:20.448778+0000","last_unstale":"2023-11-04T05:28:26.356335+0000","last_undegraded":"2023-11-04T05:28:26.356335+0000","last_fullsized":"2023-11-04T05:28:26.356335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840260+0000","last_change":"2023-11-04T05:28:21.444332+0000","last_active":"2023-11-04T05:28:25.840260+0000","last_peered":"2023-11-04T05:28:25.840260+0000","last_clean":"2023-11-04T05:28:25.840260+0000","last_became_active":"2023-11-04T05:28:21.444027+0000","last_became_peered":"2023-11-04T05:28:21.444027+0000","last_unstale":"2023-11-04T05:28:25.840260+0000","last_undegraded":"2023-11-04T05:28:25.840260+0000","last_fullsized":"2023-11-04T05:28:25.840260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742687+0000","last_change":"2023-11-04T05:28:25.846982+0000","last_active":"2023-11-04T05:28:28.742687+0000","last_peered":"2023-11-04T05:28:28.742687+0000","last_clean":"2023-11-04T05:28:28.742687+0000","last_became_active":"2023-11-04T05:28:25.846562+0000","last_became_peered":"2023-11-04T05:28:25.846562+0000","last_unstale":"2023-11-04T05:28:28.742687+0000","last_undegraded":"2023-11-04T05:28:28.742687+0000","last_fullsized":"2023-11-04T05:28:28.742687+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:52.840764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834969+0000","last_change":"2023-11-04T05:28:20.447063+0000","last_active":"2023-11-04T05:28:25.834969+0000","last_peered":"2023-11-04T05:28:25.834969+0000","last_clean":"2023-11-04T05:28:25.834969+0000","last_became_active":"2023-11-04T05:28:20.446833+0000","last_became_peered":"2023-11-04T05:28:20.446833+0000","last_unstale":"2023-11-04T05:28:25.834969+0000","last_undegraded":"2023-11-04T05:28:25.834969+0000","last_fullsized":"2023-11-04T05:28:25.834969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842874+0000","last_change":"2023-11-04T05:28:21.456075+0000","last_active":"2023-11-04T05:28:25.842874+0000","last_peered":"2023-11-04T05:28:25.842874+0000","last_clean":"2023-11-04T05:28:25.842874+0000","last_became_active":"2023-11-04T05:28:21.455801+0000","last_became_peered":"2023-11-04T05:28:21.455801+0000","last_unstale":"2023-11-04T05:28:25.842874+0000","last_undegraded":"2023-11-04T05:28:25.842874+0000","last_fullsized":"2023-11-04T05:28:25.842874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:48.444983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743366+0000","last_change":"2023-11-04T05:28:25.864053+0000","last_active":"2023-11-04T05:28:28.743366+0000","last_peered":"2023-11-04T05:28:28.743366+0000","last_clean":"2023-11-04T05:28:28.743366+0000","last_became_active":"2023-11-04T05:28:25.863669+0000","last_became_peered":"2023-11-04T05:28:25.863669+0000","last_unstale":"2023-11-04T05:28:28.743366+0000","last_undegraded":"2023-11-04T05:28:28.743366+0000","last_fullsized":"2023-11-04T05:28:28.743366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:42:30.660108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833998+0000","last_change":"2023-11-04T05:28:20.453827+0000","last_active":"2023-11-04T05:28:25.833998+0000","last_peered":"2023-11-04T05:28:25.833998+0000","last_clean":"2023-11-04T05:28:25.833998+0000","last_became_active":"2023-11-04T05:28:20.453576+0000","last_became_peered":"2023-11-04T05:28:20.453576+0000","last_unstale":"2023-11-04T05:28:25.833998+0000","last_undegraded":"2023-11-04T05:28:25.833998+0000","last_fullsized":"2023-11-04T05:28:25.833998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840329+0000","last_change":"2023-11-04T05:28:21.458086+0000","last_active":"2023-11-04T05:28:25.840329+0000","last_peered":"2023-11-04T05:28:25.840329+0000","last_clean":"2023-11-04T05:28:25.840329+0000","last_became_active":"2023-11-04T05:28:21.457739+0000","last_became_peered":"2023-11-04T05:28:21.457739+0000","last_unstale":"2023-11-04T05:28:25.840329+0000","last_undegraded":"2023-11-04T05:28:25.840329+0000","last_fullsized":"2023-11-04T05:28:25.840329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:45.088645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832505+0000","last_change":"2023-11-04T05:28:25.864371+0000","last_active":"2023-11-04T05:28:28.832505+0000","last_peered":"2023-11-04T05:28:28.832505+0000","last_clean":"2023-11-04T05:28:28.832505+0000","last_became_active":"2023-11-04T05:28:25.863374+0000","last_became_peered":"2023-11-04T05:28:25.863374+0000","last_unstale":"2023-11-04T05:28:28.832505+0000","last_undegraded":"2023-11-04T05:28:28.832505+0000","last_fullsized":"2023-11-04T05:28:28.832505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:58.207114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857793+0000","last_change":"2023-11-04T05:28:20.450039+0000","last_active":"2023-11-04T05:28:25.857793+0000","last_peered":"2023-11-04T05:28:25.857793+0000","last_clean":"2023-11-04T05:28:25.857793+0000","last_became_active":"2023-11-04T05:28:20.449761+0000","last_became_peered":"2023-11-04T05:28:20.449761+0000","last_unstale":"2023-11-04T05:28:25.857793+0000","last_undegraded":"2023-11-04T05:28:25.857793+0000","last_fullsized":"2023-11-04T05:28:25.857793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841634+0000","last_change":"2023-11-04T05:28:21.450153+0000","last_active":"2023-11-04T05:28:25.841634+0000","last_peered":"2023-11-04T05:28:25.841634+0000","last_clean":"2023-11-04T05:28:25.841634+0000","last_became_active":"2023-11-04T05:28:21.449878+0000","last_became_peered":"2023-11-04T05:28:21.449878+0000","last_unstale":"2023-11-04T05:28:25.841634+0000","last_undegraded":"2023-11-04T05:28:25.841634+0000","last_fullsized":"2023-11-04T05:28:25.841634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743302+0000","last_change":"2023-11-04T05:28:25.864439+0000","last_active":"2023-11-04T05:28:28.743302+0000","last_peered":"2023-11-04T05:28:28.743302+0000","last_clean":"2023-11-04T05:28:28.743302+0000","last_became_active":"2023-11-04T05:28:25.863909+0000","last_became_peered":"2023-11-04T05:28:25.863909+0000","last_unstale":"2023-11-04T05:28:28.743302+0000","last_undegraded":"2023-11-04T05:28:28.743302+0000","last_fullsized":"2023-11-04T05:28:28.743302+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:00:50.267120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858628+0000","last_change":"2023-11-04T05:28:20.450297+0000","last_active":"2023-11-04T05:28:25.858628+0000","last_peered":"2023-11-04T05:28:25.858628+0000","last_clean":"2023-11-04T05:28:25.858628+0000","last_became_active":"2023-11-04T05:28:20.450017+0000","last_became_peered":"2023-11-04T05:28:20.450017+0000","last_unstale":"2023-11-04T05:28:25.858628+0000","last_undegraded":"2023-11-04T05:28:25.858628+0000","last_fullsized":"2023-11-04T05:28:25.858628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:14:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840528+0000","last_change":"2023-11-04T05:28:21.457231+0000","last_active":"2023-11-04T05:28:25.840528+0000","last_peered":"2023-11-04T05:28:25.840528+0000","last_clean":"2023-11-04T05:28:25.840528+0000","last_became_active":"2023-11-04T05:28:21.456966+0000","last_became_peered":"2023-11-04T05:28:21.456966+0000","last_unstale":"2023-11-04T05:28:25.840528+0000","last_undegraded":"2023-11-04T05:28:25.840528+0000","last_fullsized":"2023-11-04T05:28:25.840528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743383+0000","last_change":"2023-11-04T05:28:25.864991+0000","last_active":"2023-11-04T05:28:28.743383+0000","last_peered":"2023-11-04T05:28:28.743383+0000","last_clean":"2023-11-04T05:28:28.743383+0000","last_became_active":"2023-11-04T05:28:25.864122+0000","last_became_peered":"2023-11-04T05:28:25.864122+0000","last_unstale":"2023-11-04T05:28:28.743383+0000","last_undegraded":"2023-11-04T05:28:28.743383+0000","last_fullsized":"2023-11-04T05:28:28.743383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:28:31.685109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834888+0000","last_change":"2023-11-04T05:28:20.449101+0000","last_active":"2023-11-04T05:28:25.834888+0000","last_peered":"2023-11-04T05:28:25.834888+0000","last_clean":"2023-11-04T05:28:25.834888+0000","last_became_active":"2023-11-04T05:28:20.448809+0000","last_became_peered":"2023-11-04T05:28:20.448809+0000","last_unstale":"2023-11-04T05:28:25.834888+0000","last_undegraded":"2023-11-04T05:28:25.834888+0000","last_fullsized":"2023-11-04T05:28:25.834888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834773+0000","last_change":"2023-11-04T05:28:21.456943+0000","last_active":"2023-11-04T05:28:25.834773+0000","last_peered":"2023-11-04T05:28:25.834773+0000","last_clean":"2023-11-04T05:28:25.834773+0000","last_became_active":"2023-11-04T05:28:21.456658+0000","last_became_peered":"2023-11-04T05:28:21.456658+0000","last_unstale":"2023-11-04T05:28:25.834773+0000","last_undegraded":"2023-11-04T05:28:25.834773+0000","last_fullsized":"2023-11-04T05:28:25.834773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838509+0000","last_change":"2023-11-04T05:28:25.865380+0000","last_active":"2023-11-04T05:28:26.838509+0000","last_peered":"2023-11-04T05:28:26.838509+0000","last_clean":"2023-11-04T05:28:26.838509+0000","last_became_active":"2023-11-04T05:28:25.865014+0000","last_became_peered":"2023-11-04T05:28:25.865014+0000","last_unstale":"2023-11-04T05:28:26.838509+0000","last_undegraded":"2023-11-04T05:28:26.838509+0000","last_fullsized":"2023-11-04T05:28:26.838509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:08.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846470+0000","last_change":"2023-11-04T05:28:20.451801+0000","last_active":"2023-11-04T05:28:25.846470+0000","last_peered":"2023-11-04T05:28:25.846470+0000","last_clean":"2023-11-04T05:28:25.846470+0000","last_became_active":"2023-11-04T05:28:20.451470+0000","last_became_peered":"2023-11-04T05:28:20.451470+0000","last_unstale":"2023-11-04T05:28:25.846470+0000","last_undegraded":"2023-11-04T05:28:25.846470+0000","last_fullsized":"2023-11-04T05:28:25.846470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:26:53.409369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834801+0000","last_change":"2023-11-04T05:28:21.465901+0000","last_active":"2023-11-04T05:28:25.834801+0000","last_peered":"2023-11-04T05:28:25.834801+0000","last_clean":"2023-11-04T05:28:25.834801+0000","last_became_active":"2023-11-04T05:28:21.465612+0000","last_became_peered":"2023-11-04T05:28:21.465612+0000","last_unstale":"2023-11-04T05:28:25.834801+0000","last_undegraded":"2023-11-04T05:28:25.834801+0000","last_fullsized":"2023-11-04T05:28:25.834801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919553+0000","last_change":"2023-11-04T05:28:25.858664+0000","last_active":"2023-11-04T05:28:27.919553+0000","last_peered":"2023-11-04T05:28:27.919553+0000","last_clean":"2023-11-04T05:28:27.919553+0000","last_became_active":"2023-11-04T05:28:25.858208+0000","last_became_peered":"2023-11-04T05:28:25.858208+0000","last_unstale":"2023-11-04T05:28:27.919553+0000","last_undegraded":"2023-11-04T05:28:27.919553+0000","last_fullsized":"2023-11-04T05:28:27.919553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:41:26.004032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858515+0000","last_change":"2023-11-04T05:28:20.443824+0000","last_active":"2023-11-04T05:28:25.858515+0000","last_peered":"2023-11-04T05:28:25.858515+0000","last_clean":"2023-11-04T05:28:25.858515+0000","last_became_active":"2023-11-04T05:28:20.443479+0000","last_became_peered":"2023-11-04T05:28:20.443479+0000","last_unstale":"2023-11-04T05:28:25.858515+0000","last_undegraded":"2023-11-04T05:28:25.858515+0000","last_fullsized":"2023-11-04T05:28:25.858515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840613+0000","last_change":"2023-11-04T05:28:21.463699+0000","last_active":"2023-11-04T05:28:25.840613+0000","last_peered":"2023-11-04T05:28:25.840613+0000","last_clean":"2023-11-04T05:28:25.840613+0000","last_became_active":"2023-11-04T05:28:21.462308+0000","last_became_peered":"2023-11-04T05:28:21.462308+0000","last_unstale":"2023-11-04T05:28:25.840613+0000","last_undegraded":"2023-11-04T05:28:25.840613+0000","last_fullsized":"2023-11-04T05:28:25.840613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836473+0000","last_change":"2023-11-04T05:28:25.849562+0000","last_active":"2023-11-04T05:28:26.836473+0000","last_peered":"2023-11-04T05:28:26.836473+0000","last_clean":"2023-11-04T05:28:26.836473+0000","last_became_active":"2023-11-04T05:28:25.849142+0000","last_became_peered":"2023-11-04T05:28:25.849142+0000","last_unstale":"2023-11-04T05:28:26.836473+0000","last_undegraded":"2023-11-04T05:28:26.836473+0000","last_fullsized":"2023-11-04T05:28:26.836473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846568+0000","last_change":"2023-11-04T05:28:20.455376+0000","last_active":"2023-11-04T05:28:25.846568+0000","last_peered":"2023-11-04T05:28:25.846568+0000","last_clean":"2023-11-04T05:28:25.846568+0000","last_became_active":"2023-11-04T05:28:20.455069+0000","last_became_peered":"2023-11-04T05:28:20.455069+0000","last_unstale":"2023-11-04T05:28:25.846568+0000","last_undegraded":"2023-11-04T05:28:25.846568+0000","last_fullsized":"2023-11-04T05:28:25.846568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:44.083516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835570+0000","last_change":"2023-11-04T05:28:21.459762+0000","last_active":"2023-11-04T05:28:25.835570+0000","last_peered":"2023-11-04T05:28:25.835570+0000","last_clean":"2023-11-04T05:28:25.835570+0000","last_became_active":"2023-11-04T05:28:21.459488+0000","last_became_peered":"2023-11-04T05:28:21.459488+0000","last_unstale":"2023-11-04T05:28:25.835570+0000","last_undegraded":"2023-11-04T05:28:25.835570+0000","last_fullsized":"2023-11-04T05:28:25.835570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836085+0000","last_change":"2023-11-04T05:28:21.458229+0000","last_active":"2023-11-04T05:28:25.836085+0000","last_peered":"2023-11-04T05:28:25.836085+0000","last_clean":"2023-11-04T05:28:25.836085+0000","last_became_active":"2023-11-04T05:28:21.457918+0000","last_became_peered":"2023-11-04T05:28:21.457918+0000","last_unstale":"2023-11-04T05:28:25.836085+0000","last_undegraded":"2023-11-04T05:28:25.836085+0000","last_fullsized":"2023-11-04T05:28:25.836085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845334+0000","last_change":"2023-11-04T05:28:21.464900+0000","last_active":"2023-11-04T05:28:25.845334+0000","last_peered":"2023-11-04T05:28:25.845334+0000","last_clean":"2023-11-04T05:28:25.845334+0000","last_became_active":"2023-11-04T05:28:21.463859+0000","last_became_peered":"2023-11-04T05:28:21.463859+0000","last_unstale":"2023-11-04T05:28:25.845334+0000","last_undegraded":"2023-11-04T05:28:25.845334+0000","last_fullsized":"2023-11-04T05:28:25.845334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839310+0000","last_change":"2023-11-04T05:28:21.466114+0000","last_active":"2023-11-04T05:28:25.839310+0000","last_peered":"2023-11-04T05:28:25.839310+0000","last_clean":"2023-11-04T05:28:25.839310+0000","last_became_active":"2023-11-04T05:28:21.465547+0000","last_became_peered":"2023-11-04T05:28:21.465547+0000","last_unstale":"2023-11-04T05:28:25.839310+0000","last_undegraded":"2023-11-04T05:28:25.839310+0000","last_fullsized":"2023-11-04T05:28:25.839310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842759+0000","last_change":"2023-11-04T05:28:21.456527+0000","last_active":"2023-11-04T05:28:25.842759+0000","last_peered":"2023-11-04T05:28:25.842759+0000","last_clean":"2023-11-04T05:28:25.842759+0000","last_became_active":"2023-11-04T05:28:21.456157+0000","last_became_peered":"2023-11-04T05:28:21.456157+0000","last_unstale":"2023-11-04T05:28:25.842759+0000","last_undegraded":"2023-11-04T05:28:25.842759+0000","last_fullsized":"2023-11-04T05:28:25.842759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:19.149382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840466+0000","last_change":"2023-11-04T05:28:21.460439+0000","last_active":"2023-11-04T05:28:25.840466+0000","last_peered":"2023-11-04T05:28:25.840466+0000","last_clean":"2023-11-04T05:28:25.840466+0000","last_became_active":"2023-11-04T05:28:21.460146+0000","last_became_peered":"2023-11-04T05:28:21.460146+0000","last_unstale":"2023-11-04T05:28:25.840466+0000","last_undegraded":"2023-11-04T05:28:25.840466+0000","last_fullsized":"2023-11-04T05:28:25.840466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858719+0000","last_change":"2023-11-04T05:28:21.447651+0000","last_active":"2023-11-04T05:28:25.858719+0000","last_peered":"2023-11-04T05:28:25.858719+0000","last_clean":"2023-11-04T05:28:25.858719+0000","last_became_active":"2023-11-04T05:28:21.447381+0000","last_became_peered":"2023-11-04T05:28:21.447381+0000","last_unstale":"2023-11-04T05:28:25.858719+0000","last_undegraded":"2023-11-04T05:28:25.858719+0000","last_fullsized":"2023-11-04T05:28:25.858719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835914+0000","last_change":"2023-11-04T05:28:21.443196+0000","last_active":"2023-11-04T05:28:25.835914+0000","last_peered":"2023-11-04T05:28:25.835914+0000","last_clean":"2023-11-04T05:28:25.835914+0000","last_became_active":"2023-11-04T05:28:21.442860+0000","last_became_peered":"2023-11-04T05:28:21.442860+0000","last_unstale":"2023-11-04T05:28:25.835914+0000","last_undegraded":"2023-11-04T05:28:25.835914+0000","last_fullsized":"2023-11-04T05:28:25.835914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841180+0000","last_change":"2023-11-04T05:28:21.455732+0000","last_active":"2023-11-04T05:28:25.841180+0000","last_peered":"2023-11-04T05:28:25.841180+0000","last_clean":"2023-11-04T05:28:25.841180+0000","last_became_active":"2023-11-04T05:28:21.455434+0000","last_became_peered":"2023-11-04T05:28:21.455434+0000","last_unstale":"2023-11-04T05:28:25.841180+0000","last_undegraded":"2023-11-04T05:28:25.841180+0000","last_fullsized":"2023-11-04T05:28:25.841180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839745+0000","last_change":"2023-11-04T05:28:21.457789+0000","last_active":"2023-11-04T05:28:25.839745+0000","last_peered":"2023-11-04T05:28:25.839745+0000","last_clean":"2023-11-04T05:28:25.839745+0000","last_became_active":"2023-11-04T05:28:21.457521+0000","last_became_peered":"2023-11-04T05:28:21.457521+0000","last_unstale":"2023-11-04T05:28:25.839745+0000","last_undegraded":"2023-11-04T05:28:25.839745+0000","last_fullsized":"2023-11-04T05:28:25.839745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841996+0000","last_change":"2023-11-04T05:28:21.462501+0000","last_active":"2023-11-04T05:28:25.841996+0000","last_peered":"2023-11-04T05:28:25.841996+0000","last_clean":"2023-11-04T05:28:25.841996+0000","last_became_active":"2023-11-04T05:28:21.462147+0000","last_became_peered":"2023-11-04T05:28:21.462147+0000","last_unstale":"2023-11-04T05:28:25.841996+0000","last_undegraded":"2023-11-04T05:28:25.841996+0000","last_fullsized":"2023-11-04T05:28:25.841996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839367+0000","last_change":"2023-11-04T05:28:21.462044+0000","last_active":"2023-11-04T05:28:25.839367+0000","last_peered":"2023-11-04T05:28:25.839367+0000","last_clean":"2023-11-04T05:28:25.839367+0000","last_became_active":"2023-11-04T05:28:21.461816+0000","last_became_peered":"2023-11-04T05:28:21.461816+0000","last_unstale":"2023-11-04T05:28:25.839367+0000","last_undegraded":"2023-11-04T05:28:25.839367+0000","last_fullsized":"2023-11-04T05:28:25.839367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843103+0000","last_change":"2023-11-04T05:28:21.458256+0000","last_active":"2023-11-04T05:28:25.843103+0000","last_peered":"2023-11-04T05:28:25.843103+0000","last_clean":"2023-11-04T05:28:25.843103+0000","last_became_active":"2023-11-04T05:28:21.457944+0000","last_became_peered":"2023-11-04T05:28:21.457944+0000","last_unstale":"2023-11-04T05:28:25.843103+0000","last_undegraded":"2023-11-04T05:28:25.843103+0000","last_fullsized":"2023-11-04T05:28:25.843103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:41:28.397824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834927+0000","last_change":"2023-11-04T05:28:21.451553+0000","last_active":"2023-11-04T05:28:25.834927+0000","last_peered":"2023-11-04T05:28:25.834927+0000","last_clean":"2023-11-04T05:28:25.834927+0000","last_became_active":"2023-11-04T05:28:21.451276+0000","last_became_peered":"2023-11-04T05:28:21.451276+0000","last_unstale":"2023-11-04T05:28:25.834927+0000","last_undegraded":"2023-11-04T05:28:25.834927+0000","last_fullsized":"2023-11-04T05:28:25.834927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":697852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":697908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":107028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.221}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":106992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-04T05:28:46.438 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-04T05:28:46.438 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-04T05:28:46.439 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-04T05:28:46.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-04T05:28:46.440 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-04T05:28:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.595+0000 7f4482c5f700 1 Processor -- start 2023-11-04T05:28:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.595+0000 7f4482c5f700 1 -- start start 2023-11-04T05:28:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f4482c5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0cf500 0x7f447c0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f4482c5f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0d14e0 con 0x7f447c0d2370 2023-11-04T05:28:46.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f4482c5f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0d1780 con 0x7f447c15da80 2023-11-04T05:28:46.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f4482c5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0d2820 con 0x7f447c0cf500 2023-11-04T05:28:46.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f44811fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f447c0d2370 0x7f447c06c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57292/0 (socket says 172.21.15.3:57292) 2023-11-04T05:28:46.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f44811fc700 1 -- 172.21.15.3:0/61358825 learned_addr learned my addr 172.21.15.3:0/61358825 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447bfff700 1 --2- 172.21.15.3:0/61358825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0cf500 0x7f447c0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615268328 0 0) 0x7f447c0d14e0 con 0x7f447c0d2370 2023-11-04T05:28:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f446c0035b0 con 0x7f447c0d2370 2023-11-04T05:28:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120828652 0 0) 0x7f447c0d1780 con 0x7f447c15da80 2023-11-04T05:28:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447c0d14e0 con 0x7f447c15da80 2023-11-04T05:28:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447bfff700 1 -- 172.21.15.3:0/61358825 >> v1:172.21.15.188:6789/0 conn(0x7f447c15da80 legacy=0x7f447c0d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447bfff700 1 -- 172.21.15.3:0/61358825 >> v1:172.21.15.3:6789/0 conn(0x7f447c0d2370 legacy=0x7f447c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447bfff700 1 -- 172.21.15.3:0/61358825 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0d1a60 con 0x7f447c0cf500 2023-11-04T05:28:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.596+0000 7f447bfff700 1 --2- 172.21.15.3:0/61358825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0cf500 0x7f447c0d1e30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4468009870 tx=0x7f4468009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d654f65d6ea47bde server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468011830 con 0x7f447c0cf500 2023-11-04T05:28:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468011990 con 0x7f447c0cf500 2023-11-04T05:28:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f447b7fe700 1 -- 172.21.15.3:0/61358825 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468011d20 con 0x7f447c0cf500 2023-11-04T05:28:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 -- 172.21.15.3:0/61358825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0cf500 msgr2=0x7f447c0d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 --2- 172.21.15.3:0/61358825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0cf500 0x7f447c0d1e30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4468009870 tx=0x7f4468009b80 comp rx=0 tx=0).stop 2023-11-04T05:28:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 -- 172.21.15.3:0/61358825 shutdown_connections 2023-11-04T05:28:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 --2- 172.21.15.3:0/61358825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0cf500 0x7f447c0d1e30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 -- 172.21.15.3:0/61358825 >> 172.21.15.3:0/61358825 conn(0x7f447c08aff0 msgr2=0x7f447c0d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 -- 172.21.15.3:0/61358825 shutdown_connections 2023-11-04T05:28:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.597+0000 7f4482c5f700 1 -- 172.21.15.3:0/61358825 wait complete. 2023-11-04T05:28:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.598+0000 7f4482c5f700 1 Processor -- start 2023-11-04T05:28:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 -- start start 2023-11-04T05:28:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 0x7f447c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0d2370 0x7f447c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f447c15da80 0x7f447c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c15a440 con 0x7f447c0cf500 2023-11-04T05:28:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0cbbf0 con 0x7f447c15da80 2023-11-04T05:28:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f4482c5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c15ea30 con 0x7f447c0d2370 2023-11-04T05:28:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.599+0000 7f44809fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 0x7f447c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f447bfff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0d2370 0x7f447c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 0x7f447c159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51946/0 (socket says 172.21.15.3:51946) 2023-11-04T05:28:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 -- 172.21.15.3:0/477649789 learned_addr learned my addr 172.21.15.3:0/477649789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44811fc700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f447c15da80 0x7f447c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 -- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0d2370 msgr2=0x7f447c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0d2370 0x7f447c15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 -- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f447c15da80 msgr2=0x7f447c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f447c15da80 0x7f447c0b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 -- 172.21.15.3:0/477649789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44680094a0 con 0x7f447c0cf500 2023-11-04T05:28:46.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44809fb700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 0x7f447c159f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f447000c4d0 tx=0x7f447000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700052d0 con 0x7f447c0cf500 2023-11-04T05:28:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470005430 con 0x7f447c0cf500 2023-11-04T05:28:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700038f0 con 0x7f447c0cf500 2023-11-04T05:28:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c15ed10 con 0x7f447c0cf500 2023-11-04T05:28:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.600+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c15f2d0 con 0x7f447c0cf500 2023-11-04T05:28:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.601+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c074df0 con 0x7f447c0cf500 2023-11-04T05:28:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.602+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4470003a50 con 0x7f447c0cf500 2023-11-04T05:28:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.602+0000 7f44797fa700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f446006af00 0x7f446006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.602+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4470088bc0 con 0x7f447c0cf500 2023-11-04T05:28:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.602+0000 7f447bfff700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f446006af00 0x7f446006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:46.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.603+0000 7f447bfff700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f446006af00 0x7f446006d3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f447c0cf940 tx=0x7f446801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:46.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.607+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4470053a10 con 0x7f447c0cf500 2023-11-04T05:28:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.865+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f447c06c970 con 0x7f447c0cf500 2023-11-04T05:28:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.869+0000 7f44797fa700 1 -- 172.21.15.3:0/477649789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7f4470017090 con 0x7f447c0cf500 2023-11-04T05:28:46.873 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f446006af00 msgr2=0x7f446006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f446006af00 0x7f446006d3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f447c0cf940 tx=0x7f446801c040 comp rx=0 tx=0).stop 2023-11-04T05:28:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 msgr2=0x7f447c159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 0x7f447c159f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f447000c4d0 tx=0x7f447000c7e0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 shutdown_connections 2023-11-04T05:28:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f446006af00 0x7f446006d3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f447c15da80 0x7f447c0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f447c0cf500 0x7f447c159f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 --2- 172.21.15.3:0/477649789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f447c0d2370 0x7f447c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 >> 172.21.15.3:0/477649789 conn(0x7f447c08aff0 msgr2=0x7f447c0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 shutdown_connections 2023-11-04T05:28:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:46.871+0000 7f4482c5f700 1 -- 172.21.15.3:0/477649789 wait complete. 2023-11-04T05:28:46.885 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3102759174},{"type":"v1","addr":"172.21.15.188:6833","nonce":3102759174}]},"active_addr":"172.21.15.188:6833/3102759174","active_change":"2023-11-04T05:28:03.737998+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":1572453480}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":2351281738}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":2495479409}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":3454777187}]}]} 2023-11-04T05:28:46.889 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-04T05:28:46.890 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-04T05:28:46.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.013+0000 7f388cfaf700 1 Processor -- start 2023-11-04T05:28:47.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.013+0000 7f388cfaf700 1 -- start start 2023-11-04T05:28:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f388cfaf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f388815bd70 0x7f38880cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f388cfaf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38880b75f0 con 0x7f3888152aa0 2023-11-04T05:28:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f388cfaf700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388815c250 con 0x7f38881526c0 2023-11-04T05:28:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f388cfaf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388815cdd0 con 0x7f388815bd70 2023-11-04T05:28:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f3886d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f388815bd70 0x7f38880cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f3885d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3888152aa0 0x7f38880cde20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57302/0 (socket says 172.21.15.3:57302) 2023-11-04T05:28:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.014+0000 7f3885d9b700 1 -- 172.21.15.3:0/1313490666 learned_addr learned my addr 172.21.15.3:0/1313490666 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f3886d9d700 1 -- 172.21.15.3:0/1313490666 >> v1:172.21.15.188:6789/0 conn(0x7f38881526c0 legacy=0x7f38880b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f3886d9d700 1 -- 172.21.15.3:0/1313490666 >> v1:172.21.15.3:6789/0 conn(0x7f3888152aa0 legacy=0x7f38880cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f3886d9d700 1 -- 172.21.15.3:0/1313490666 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388815c530 con 0x7f388815bd70 2023-11-04T05:28:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f388559a700 1 -- 172.21.15.3:0/1313490666 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246997665 0 0) 0x7f38880b75f0 con 0x7f3888152aa0 2023-11-04T05:28:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f3886d9d700 1 --2- 172.21.15.3:0/1313490666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388815bd70 0x7f38880cbcb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3880009d90 tx=0x7f388000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bb39a9eea410c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f388559a700 1 -- 172.21.15.3:0/1313490666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388000e070 con 0x7f388815bd70 2023-11-04T05:28:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f388559a700 1 -- 172.21.15.3:0/1313490666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388000bbd0 con 0x7f388815bd70 2023-11-04T05:28:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.015+0000 7f388559a700 1 -- 172.21.15.3:0/1313490666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880012430 con 0x7f388815bd70 2023-11-04T05:28:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 -- 172.21.15.3:0/1313490666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388815bd70 msgr2=0x7f38880cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 --2- 172.21.15.3:0/1313490666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388815bd70 0x7f38880cbcb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3880009d90 tx=0x7f388000b330 comp rx=0 tx=0).stop 2023-11-04T05:28:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 -- 172.21.15.3:0/1313490666 shutdown_connections 2023-11-04T05:28:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 --2- 172.21.15.3:0/1313490666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f388815bd70 0x7f38880cbcb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 -- 172.21.15.3:0/1313490666 >> 172.21.15.3:0/1313490666 conn(0x7f388806ab20 msgr2=0x7f388806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 -- 172.21.15.3:0/1313490666 shutdown_connections 2023-11-04T05:28:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.016+0000 7f388cfaf700 1 -- 172.21.15.3:0/1313490666 wait complete. 2023-11-04T05:28:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.017+0000 7f388cfaf700 1 Processor -- start 2023-11-04T05:28:47.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.017+0000 7f388cfaf700 1 -- start start 2023-11-04T05:28:47.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388cfaf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 0x7f38880cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388cfaf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f388815bd70 0x7f3888149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388cfaf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3888155580 0x7f388815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388cfaf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388806bc30 con 0x7f38881526c0 2023-11-04T05:28:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388cfaf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388806bf10 con 0x7f388815bd70 2023-11-04T05:28:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388cfaf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388806c1f0 con 0x7f3888155580 2023-11-04T05:28:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 0x7f38880cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 0x7f38880cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51960/0 (socket says 172.21.15.3:51960) 2023-11-04T05:28:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f388659c700 1 -- 172.21.15.3:0/2801659281 learned_addr learned my addr 172.21.15.3:0/2801659281 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.018+0000 7f3885d9b700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f388815bd70 0x7f3888149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f3886d9d700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3888155580 0x7f388815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388659c700 1 -- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3888155580 msgr2=0x7f388815a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388659c700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3888155580 0x7f388815a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388659c700 1 -- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f388815bd70 msgr2=0x7f3888149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388659c700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f388815bd70 0x7f3888149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388659c700 1 -- 172.21.15.3:0/2801659281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880009a70 con 0x7f38881526c0 2023-11-04T05:28:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f3885d9b700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f388815bd70 0x7f3888149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:47.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388659c700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 0x7f38880cf170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f387800c4e0 tx=0x7f387800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38780052d0 con 0x7f38881526c0 2023-11-04T05:28:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878005430 con 0x7f38881526c0 2023-11-04T05:28:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388806c4d0 con 0x7f38881526c0 2023-11-04T05:28:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38780038c0 con 0x7f38881526c0 2023-11-04T05:28:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.019+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388806ca90 con 0x7f38881526c0 2023-11-04T05:28:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.020+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3888074df0 con 0x7f38881526c0 2023-11-04T05:28:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.021+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3878003a20 con 0x7f38881526c0 2023-11-04T05:28:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.021+0000 7f387effd700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f386806af00 0x7f386806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.021+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3878088da0 con 0x7f38881526c0 2023-11-04T05:28:47.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.021+0000 7f3885d9b700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f386806af00 0x7f386806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.022+0000 7f3885d9b700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f386806af00 0x7f386806d3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38880d2200 tx=0x7f3870007ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.026+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f3878053b60 con 0x7f38881526c0 2023-11-04T05:28:47.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.226+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38880b3e80 con 0x7f38881526c0 2023-11-04T05:28:47.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.227+0000 7f387effd700 1 -- 172.21.15.3:0/2801659281 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f3878017090 con 0x7f38881526c0 2023-11-04T05:28:47.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:47.230 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:47.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f386806af00 msgr2=0x7f386806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f386806af00 0x7f386806d3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38880d2200 tx=0x7f3870007ce0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 msgr2=0x7f38880cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 0x7f38880cf170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f387800c4e0 tx=0x7f387800c8a0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 shutdown_connections 2023-11-04T05:28:47.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f386806af00 0x7f386806d3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3888155580 0x7f388815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38881526c0 0x7f38880cf170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 --2- 172.21.15.3:0/2801659281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f388815bd70 0x7f3888149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 >> 172.21.15.3:0/2801659281 conn(0x7f388806ab20 msgr2=0x7f38880b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 shutdown_connections 2023-11-04T05:28:47.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.229+0000 7f388cfaf700 1 -- 172.21.15.3:0/2801659281 wait complete. 2023-11-04T05:28:47.244 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-04T05:28:47.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:47.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.367+0000 7f89e8a8a700 1 Processor -- start 2023-11-04T05:28:47.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.367+0000 7f89e8a8a700 1 -- start start 2023-11-04T05:28:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e8a8a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89e415e350 0x7f89e406dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e8a8a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e4154930 con 0x7f89e4154550 2023-11-04T05:28:47.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e8a8a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e415e9b0 con 0x7f89e406d790 2023-11-04T05:28:47.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e8a8a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e415e7a0 con 0x7f89e415e350 2023-11-04T05:28:47.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f89e4154550 0x7f89e40cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57316/0 (socket says 172.21.15.3:57316) 2023-11-04T05:28:47.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e259c700 1 -- 172.21.15.3:0/2315755847 learned_addr learned my addr 172.21.15.3:0/2315755847 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:47.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e2d9d700 1 --2- 172.21.15.3:0/2315755847 >> v2:172.21.15.188:3300/0 conn(0x7f89e415e350 0x7f89e406dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917699528 0 0) 0x7f89e4154930 con 0x7f89e4154550 2023-11-04T05:28:47.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.368+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89cc0046c0 con 0x7f89e4154550 2023-11-04T05:28:47.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 503275782 0 0) 0x7f89cc0046c0 con 0x7f89e4154550 2023-11-04T05:28:47.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 >> v2:172.21.15.188:3300/0 conn(0x7f89e415e350 msgr2=0x7f89e406dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 --2- 172.21.15.3:0/2315755847 >> v2:172.21.15.188:3300/0 conn(0x7f89e415e350 0x7f89e406dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 >> v1:172.21.15.188:6789/0 conn(0x7f89e406d790 legacy=0x7f89e415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40cddf0 con 0x7f89e4154550 2023-11-04T05:28:47.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f89d8001670 con 0x7f89e4154550 2023-11-04T05:28:47.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 >> v1:172.21.15.3:6789/0 conn(0x7f89e4154550 legacy=0x7f89e40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89cc004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89cc005510 0x7f89cc0059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89cc006280 0x7f89cc00cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e415e7a0 con 0x7f89cc006280 2023-11-04T05:28:47.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc00d1c0 con 0x7f89cc004b10 2023-11-04T05:28:47.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc00d380 con 0x7f89cc005510 2023-11-04T05:28:47.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e259c700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89cc004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e2d9d700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89cc006280 0x7f89cc00cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e1d9b700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89cc005510 0x7f89cc0059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e259c700 1 -- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89cc005510 msgr2=0x7f89cc0059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e259c700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89cc005510 0x7f89cc0059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e259c700 1 -- 172.21.15.3:0/2315755847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89cc006280 msgr2=0x7f89cc00cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e259c700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89cc006280 0x7f89cc00cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e259c700 1 -- 172.21.15.3:0/2315755847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0046c0 con 0x7f89cc004b10 2023-11-04T05:28:47.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e2d9d700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89cc006280 0x7f89cc00cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:47.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.369+0000 7f89e2d9d700 1 -- 172.21.15.3:0/2315755847 reap_dead start 2023-11-04T05:28:47.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e259c700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89cc004ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c2f0 tx=0x7f89cc006990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c705ef9b915c117 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d800ce30 con 0x7f89cc004b10 2023-11-04T05:28:47.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8005960 con 0x7f89cc004b10 2023-11-04T05:28:47.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e159a700 1 -- 172.21.15.3:0/2315755847 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8005cf0 con 0x7f89cc004b10 2023-11-04T05:28:47.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 -- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 msgr2=0x7f89cc004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89cc004ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c2f0 tx=0x7f89cc006990 comp rx=0 tx=0).stop 2023-11-04T05:28:47.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 -- 172.21.15.3:0/2315755847 shutdown_connections 2023-11-04T05:28:47.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/2315755847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89cc004ef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 -- 172.21.15.3:0/2315755847 >> 172.21.15.3:0/2315755847 conn(0x7f89e40cfd50 msgr2=0x7f89e40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 -- 172.21.15.3:0/2315755847 shutdown_connections 2023-11-04T05:28:47.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.370+0000 7f89e8a8a700 1 -- 172.21.15.3:0/2315755847 wait complete. 2023-11-04T05:28:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.372+0000 7f89e8a8a700 1 Processor -- start 2023-11-04T05:28:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.372+0000 7f89e8a8a700 1 -- start start 2023-11-04T05:28:47.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e8a8a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89e4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e8a8a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89e4156b40 0x7f89e4156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e8a8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 0x7f89e40c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e8a8a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4157500 con 0x7f89e40c7b90 2023-11-04T05:28:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e8a8a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41f1b00 con 0x7f89cc004b10 2023-11-04T05:28:47.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e8a8a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40c8930 con 0x7f89e4156b40 2023-11-04T05:28:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 0x7f89e40c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 0x7f89e40c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51984/0 (socket says 172.21.15.3:51984) 2023-11-04T05:28:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e2d9d700 1 -- 172.21.15.3:0/1209864885 learned_addr learned my addr 172.21.15.3:0/1209864885 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e1d9b700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89e4156b40 0x7f89e4156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.373+0000 7f89e259c700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89e4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e2d9d700 1 -- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89e4156b40 msgr2=0x7f89e4156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e2d9d700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89e4156b40 0x7f89e4156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e2d9d700 1 -- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 msgr2=0x7f89e4152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e2d9d700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89e4152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e2d9d700 1 -- 172.21.15.3:0/1209864885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40c8c10 con 0x7f89e40c7b90 2023-11-04T05:28:47.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e259c700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89e4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:47.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e2d9d700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 0x7f89e40c8010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89e415e730 tx=0x7f89dc00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc01c070 con 0x7f89e40c7b90 2023-11-04T05:28:47.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00f040 con 0x7f89e40c7b90 2023-11-04T05:28:47.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e41f1d50 con 0x7f89e40c7b90 2023-11-04T05:28:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00e040 con 0x7f89e40c7b90 2023-11-04T05:28:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.374+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e41527c0 con 0x7f89e40c7b90 2023-11-04T05:28:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.375+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e41f2270 con 0x7f89e40c7b90 2023-11-04T05:28:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.376+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89dc015430 con 0x7f89e40c7b90 2023-11-04T05:28:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.376+0000 7f89d2ffd700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f89c406af00 0x7f89c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.376+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f89dc089560 con 0x7f89e40c7b90 2023-11-04T05:28:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.376+0000 7f89e259c700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f89c406af00 0x7f89c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.377+0000 7f89e259c700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f89c406af00 0x7f89c406d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89d8015040 tx=0x7f89d8007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.381+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f89e41f2270 con 0x7f89e40c7b90 2023-11-04T05:28:47.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.583+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e41f2270 con 0x7f89e40c7b90 2023-11-04T05:28:47.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.584+0000 7f89d2ffd700 1 -- 172.21.15.3:0/1209864885 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7f89e41f2270 con 0x7f89e40c7b90 2023-11-04T05:28:47.586 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:47.587 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:47.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f89c406af00 msgr2=0x7f89c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f89c406af00 0x7f89c406d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89d8015040 tx=0x7f89d8007000 comp rx=0 tx=0).stop 2023-11-04T05:28:47.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 msgr2=0x7f89e40c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 0x7f89e40c8010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89e415e730 tx=0x7f89dc00f490 comp rx=0 tx=0).stop 2023-11-04T05:28:47.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 shutdown_connections 2023-11-04T05:28:47.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f89c406af00 0x7f89c406d3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89cc004b10 0x7f89e4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89e40c7b90 0x7f89e40c8010 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 --2- 172.21.15.3:0/1209864885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89e4156b40 0x7f89e4156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 >> 172.21.15.3:0/1209864885 conn(0x7f89e40cfd50 msgr2=0x7f89e415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 shutdown_connections 2023-11-04T05:28:47.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.586+0000 7f89e8a8a700 1 -- 172.21.15.3:0/1209864885 wait complete. 2023-11-04T05:28:47.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-04T05:28:47.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-04T05:28:47.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-04T05:28:47.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-04T05:28:47.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-04T05:28:47.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-04T05:28:47.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-04T05:28:47.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-04T05:28:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.820+0000 7f3197f8d700 1 Processor -- start 2023-11-04T05:28:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.821+0000 7f3197f8d700 1 -- start start 2023-11-04T05:28:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3197f8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f319006eeb0 0x7f31900cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3197f8d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3190157500 0x7f3190157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3197f8d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f319015d930 0x7f319015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3197f8d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319015e300 con 0x7f319006eeb0 2023-11-04T05:28:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3197f8d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3190160ca0 con 0x7f319015d930 2023-11-04T05:28:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3197f8d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3190071c90 con 0x7f3190157500 2023-11-04T05:28:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.822+0000 7f3195d29700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f319006eeb0 0x7f31900cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f319006eeb0 0x7f31900cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51990/0 (socket says 172.21.15.3:51990) 2023-11-04T05:28:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 -- 172.21.15.3:0/3831057693 learned_addr learned my addr 172.21.15.3:0/3831057693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f319652a700 1 --2- 172.21.15.3:0/3831057693 >> v2:172.21.15.188:3300/0 conn(0x7f319015d930 0x7f319015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 -- 172.21.15.3:0/3831057693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3190157500 msgr2=0x7f3190157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 --2- 172.21.15.3:0/3831057693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3190157500 0x7f3190157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 -- 172.21.15.3:0/3831057693 >> v2:172.21.15.188:3300/0 conn(0x7f319015d930 msgr2=0x7f319015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 --2- 172.21.15.3:0/3831057693 >> v2:172.21.15.188:3300/0 conn(0x7f319015d930 0x7f319015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 -- 172.21.15.3:0/3831057693 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3190071f70 con 0x7f319006eeb0 2023-11-04T05:28:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f3195d29700 1 --2- 172.21.15.3:0/3831057693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319006eeb0 0x7f31900cc760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3190070e80 tx=0x7f3180009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7b802c2e4ca0e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.824+0000 7f57f1ecc700 1 Processor -- start 2023-11-04T05:28:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.824+0000 7f57f1ecc700 1 -- start start 2023-11-04T05:28:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f57f1ecc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57ec15d930 0x7f57ec15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f57f1ecc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57ec0c9b40 0x7f57ec0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f57f1ecc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec157d80 con 0x7f57ec1579a0 2023-11-04T05:28:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f57f1ecc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec15e300 con 0x7f57ec15d930 2023-11-04T05:28:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f57f1ecc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0729a0 con 0x7f57ec0c9b40 2023-11-04T05:28:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eb7fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f57ec1579a0 0x7f57ec0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:35884/0 (socket says 172.21.15.3:35884) 2023-11-04T05:28:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57ec15d930 0x7f57ec15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eb7fe700 1 -- 172.21.15.3:0/2099557765 learned_addr learned my addr 172.21.15.3:0/2099557765 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:47.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f3194d27700 1 -- 172.21.15.3:0/3831057693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000b870 con 0x7f319006eeb0 2023-11-04T05:28:47.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.821+0000 7f262067f700 1 Processor -- start 2023-11-04T05:28:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.821+0000 7f262067f700 1 -- start start 2023-11-04T05:28:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f262067f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2618157b50 0x7f261815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f262067f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f261815de70 0x7f26180c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f262067f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180719b0 con 0x7f2618157770 2023-11-04T05:28:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f262067f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261815e300 con 0x7f2618157b50 2023-11-04T05:28:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f262067f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2618071c90 con 0x7f261815de70 2023-11-04T05:28:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f261ec1c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f261815de70 0x7f26180c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f261ec1c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f261815de70 0x7f26180c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51082/0 (socket says 172.21.15.3:51082) 2023-11-04T05:28:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.823+0000 7f261ec1c700 1 -- 172.21.15.3:0/865530599 learned_addr learned my addr 172.21.15.3:0/865530599 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f261ec1c700 1 -- 172.21.15.3:0/865530599 >> v2:172.21.15.3:3300/0 conn(0x7f2618157b50 msgr2=0x7f261815d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f261ec1c700 1 --2- 172.21.15.3:0/865530599 >> v2:172.21.15.3:3300/0 conn(0x7f2618157b50 0x7f261815d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f261ec1c700 1 -- 172.21.15.3:0/865530599 >> v1:172.21.15.188:6789/0 conn(0x7f2618157770 legacy=0x7f26180cc760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f261ec1c700 1 -- 172.21.15.3:0/865530599 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618071f70 con 0x7f261815de70 2023-11-04T05:28:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.825+0000 7f261ec1c700 1 --2- 172.21.15.3:0/865530599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 0x7f26180c9b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26180cb9d0 tx=0x7f2608009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96ea43698071cd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f3194d27700 1 -- 172.21.15.3:0/3831057693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000b9d0 con 0x7f319006eeb0 2023-11-04T05:28:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f3194d27700 1 -- 172.21.15.3:0/3831057693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000bd40 con 0x7f319006eeb0 2023-11-04T05:28:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57ebfff700 1 --2- 172.21.15.3:0/2099557765 >> v2:172.21.15.188:3300/0 conn(0x7f57ec0c9b40 0x7f57ec0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eaffd700 1 -- 172.21.15.3:0/2099557765 >> v2:172.21.15.188:3300/0 conn(0x7f57ec0c9b40 msgr2=0x7f57ec0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eaffd700 1 --2- 172.21.15.3:0/2099557765 >> v2:172.21.15.188:3300/0 conn(0x7f57ec0c9b40 0x7f57ec0c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eaffd700 1 -- 172.21.15.3:0/2099557765 >> v1:172.21.15.188:6789/0 conn(0x7f57ec1579a0 legacy=0x7f57ec0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eaffd700 1 -- 172.21.15.3:0/2099557765 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec071900 con 0x7f57ec15d930 2023-11-04T05:28:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.826+0000 7f57eaffd700 1 --2- 172.21.15.3:0/2099557765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec15d930 0x7f57ec15ddc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0c9a90 tx=0x7f57e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290bc4ae630cd358 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f261d419700 1 -- 172.21.15.3:0/865530599 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608004230 con 0x7f261815de70 2023-11-04T05:28:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f261d419700 1 -- 172.21.15.3:0/865530599 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608004390 con 0x7f261815de70 2023-11-04T05:28:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f261d419700 1 -- 172.21.15.3:0/865530599 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26080036a0 con 0x7f261815de70 2023-11-04T05:28:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 -- 172.21.15.3:0/3831057693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319006eeb0 msgr2=0x7f31900cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 --2- 172.21.15.3:0/3831057693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319006eeb0 0x7f31900cc760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3190070e80 tx=0x7f3180009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f57ea7fc700 1 -- 172.21.15.3:0/2099557765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000f8a0 con 0x7f57ec15d930 2023-11-04T05:28:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f57ea7fc700 1 -- 172.21.15.3:0/2099557765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000fa00 con 0x7f57ec15d930 2023-11-04T05:28:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f57ea7fc700 1 -- 172.21.15.3:0/2099557765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000fd90 con 0x7f57ec15d930 2023-11-04T05:28:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 -- 172.21.15.3:0/3831057693 shutdown_connections 2023-11-04T05:28:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 --2- 172.21.15.3:0/3831057693 >> v2:172.21.15.188:3300/0 conn(0x7f319015d930 0x7f319015ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 --2- 172.21.15.3:0/3831057693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319006eeb0 0x7f31900cc760 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 --2- 172.21.15.3:0/3831057693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3190157500 0x7f3190157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.827+0000 7f3197f8d700 1 -- 172.21.15.3:0/3831057693 >> 172.21.15.3:0/3831057693 conn(0x7f319006b4d0 msgr2=0x7f319006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7fdbf2dbf700 1 Processor -- start 2023-11-04T05:28:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 -- 172.21.15.3:0/2099557765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec15d930 msgr2=0x7f57ec15ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 --2- 172.21.15.3:0/2099557765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec15d930 0x7f57ec15ddc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0c9a90 tx=0x7f57e0009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f3197f8d700 1 -- 172.21.15.3:0/3831057693 shutdown_connections 2023-11-04T05:28:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f262067f700 1 -- 172.21.15.3:0/865530599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 msgr2=0x7f26180c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f262067f700 1 --2- 172.21.15.3:0/865530599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 0x7f26180c9b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26180cb9d0 tx=0x7f2608009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f262067f700 1 -- 172.21.15.3:0/865530599 shutdown_connections 2023-11-04T05:28:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f262067f700 1 --2- 172.21.15.3:0/865530599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 0x7f26180c9b40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f262067f700 1 --2- 172.21.15.3:0/865530599 >> v2:172.21.15.3:3300/0 conn(0x7f2618157b50 0x7f261815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f262067f700 1 -- 172.21.15.3:0/865530599 >> 172.21.15.3:0/865530599 conn(0x7f261806b4d0 msgr2=0x7f261806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 -- 172.21.15.3:0/2099557765 shutdown_connections 2023-11-04T05:28:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 --2- 172.21.15.3:0/2099557765 >> v2:172.21.15.188:3300/0 conn(0x7f57ec0c9b40 0x7f57ec0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 --2- 172.21.15.3:0/2099557765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec15d930 0x7f57ec15ddc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 -- 172.21.15.3:0/2099557765 >> 172.21.15.3:0/2099557765 conn(0x7f57ec06b150 msgr2=0x7f57ec06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7fdbf2dbf700 1 -- start start 2023-11-04T05:28:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 -- 172.21.15.3:0/2099557765 shutdown_connections 2023-11-04T05:28:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.828+0000 7f57f1ecc700 1 -- 172.21.15.3:0/2099557765 wait complete. 2023-11-04T05:28:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf2dbf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbec071f00 0x7fdbec0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf2dbf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0cddb0 0x7fdbec06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf2dbf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0cb7a0 con 0x7fdbec15a200 2023-11-04T05:28:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf2dbf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec06c290 con 0x7fdbec071f00 2023-11-04T05:28:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf2dbf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec072670 con 0x7fdbec0cddb0 2023-11-04T05:28:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf0b5b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdbec15a200 0x7fdbec0cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57350/0 (socket says 172.21.15.3:57350) 2023-11-04T05:28:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbf0b5b700 1 -- 172.21.15.3:0/2102848066 learned_addr learned my addr 172.21.15.3:0/2102848066 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbebfff700 1 --2- 172.21.15.3:0/2102848066 >> v2:172.21.15.3:3300/0 conn(0x7fdbec071f00 0x7fdbec0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226220639 0 0) 0x7fdbec0cb7a0 con 0x7fdbec15a200 2023-11-04T05:28:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd40057c0 con 0x7fdbec15a200 2023-11-04T05:28:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.829+0000 7f3197f8d700 1 -- 172.21.15.3:0/3831057693 wait complete. 2023-11-04T05:28:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7f262067f700 1 -- 172.21.15.3:0/865530599 shutdown_connections 2023-11-04T05:28:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740416230 0 0) 0x7fdbd40057c0 con 0x7fdbec15a200 2023-11-04T05:28:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0cddb0 msgr2=0x7fdbec06ab20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-04T05:28:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0cddb0 0x7fdbec06ab20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 >> v2:172.21.15.3:3300/0 conn(0x7fdbec071f00 msgr2=0x7fdbec0cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 --2- 172.21.15.3:0/2102848066 >> v2:172.21.15.3:3300/0 conn(0x7fdbec071f00 0x7fdbec0cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0cb150 con 0x7fdbec15a200 2023-11-04T05:28:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fdbe40019f0 con 0x7fdbec15a200 2023-11-04T05:28:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbebfff700 1 --2- 172.21.15.3:0/2102848066 >> v2:172.21.15.3:3300/0 conn(0x7fdbec071f00 0x7fdbec0cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 >> v1:172.21.15.3:6789/0 conn(0x7fdbec15a200 legacy=0x7fdbec0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0cddb0 0x7fdbd4005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd4006170 0x7fdbd4006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd4006fa0 0x7fdbd4007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec06c290 con 0x7fdbd4006170 2023-11-04T05:28:47.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec072670 con 0x7fdbd4006fa0 2023-11-04T05:28:47.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4010020 con 0x7fdbec0cddb0 2023-11-04T05:28:47.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7f57f1ecc700 1 Processor -- start 2023-11-04T05:28:47.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbebfff700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd4006170 0x7fdbd4006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbf135c700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd4006fa0 0x7fdbd4007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbebfff700 1 -- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0cddb0 msgr2=0x7fdbd4005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-04T05:28:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbebfff700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0cddb0 0x7fdbd4005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbebfff700 1 -- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd4006fa0 msgr2=0x7fdbd4007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbebfff700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd4006fa0 0x7fdbd4007420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7fdbebfff700 1 -- 172.21.15.3:0/2102848066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0cb150 con 0x7fdbd4006170 2023-11-04T05:28:47.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7fdbebfff700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd4006170 0x7fdbd4006a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009800 tx=0x7fdbd4007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ad588bf86b945b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7f57f1ecc700 1 -- start start 2023-11-04T05:28:47.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7f3197f8d700 1 Processor -- start 2023-11-04T05:28:47.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.831+0000 7f3197f8d700 1 -- start start 2023-11-04T05:28:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.830+0000 7f262067f700 1 -- 172.21.15.3:0/865530599 wait complete. 2023-11-04T05:28:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57f1ecc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec0c9b40 0x7f57ec14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57f1ecc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57ec1579a0 0x7f57ec0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57f1ecc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ec15d930 0x7f57ec14ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57f1ecc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0b0240 con 0x7f57ec0c9b40 2023-11-04T05:28:47.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57f1ecc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0af2f0 con 0x7f57ec1579a0 2023-11-04T05:28:47.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57f1ecc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0af5c0 con 0x7f57ec15d930 2023-11-04T05:28:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f262067f700 1 Processor -- start 2023-11-04T05:28:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f262067f700 1 -- start start 2023-11-04T05:28:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec0c9b40 0x7f57ec14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57eb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec0c9b40 0x7f57ec14bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52016/0 (socket says 172.21.15.3:52016) 2023-11-04T05:28:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57eb7fe700 1 -- 172.21.15.3:0/479088342 learned_addr learned my addr 172.21.15.3:0/479088342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f57eaffd700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57ec1579a0 0x7f57ec0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3197f8d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319006eeb0 0x7f31900c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3197f8d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190157500 0x7f31900c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3197f8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319015d930 0x7f31900ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3197f8d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31900c9010 con 0x7f319015d930 2023-11-04T05:28:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3197f8d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31900c9520 con 0x7f3190157500 2023-11-04T05:28:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3197f8d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31900c9240 con 0x7f319006eeb0 2023-11-04T05:28:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0004230 con 0x7fdbd4006170 2023-11-04T05:28:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000d040 con 0x7fdbd4006170 2023-11-04T05:28:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7fdbeb7fe700 1 -- 172.21.15.3:0/2102848066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00045b0 con 0x7fdbd4006170 2023-11-04T05:28:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f57eb7fe700 1 -- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ec15d930 msgr2=0x7f57ec14ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f57eb7fe700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ec15d930 0x7f57ec14ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f57eb7fe700 1 -- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57ec1579a0 msgr2=0x7f57ec0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f57eb7fe700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57ec1579a0 0x7f57ec0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f57eb7fe700 1 -- 172.21.15.3:0/479088342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00094a0 con 0x7f57ec0c9b40 2023-11-04T05:28:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.832+0000 7f3195528700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190157500 0x7f31900c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319006eeb0 0x7f31900c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195528700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190157500 0x7f31900c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51108/0 (socket says 172.21.15.3:51108) 2023-11-04T05:28:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319006eeb0 0x7f31900c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43078/0 (socket says 172.21.15.3:43078) 2023-11-04T05:28:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195528700 1 -- 172.21.15.3:0/2577430771 learned_addr learned my addr 172.21.15.3:0/2577430771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f319652a700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319015d930 0x7f31900ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 -- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190157500 msgr2=0x7f31900c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3190157500 0x7f31900c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 -- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319015d930 msgr2=0x7f31900ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319015d930 0x7f31900ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 -- 172.21.15.3:0/2577430771 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180009550 con 0x7f319006eeb0 2023-11-04T05:28:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f3195d29700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319006eeb0 0x7f31900c3a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3180016000 tx=0x7f3180004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f262067f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618157770 0x7f26180b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f262067f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2618157b50 0x7f26180c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f262067f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 0x7f26180c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f262067f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26180c7db0 con 0x7f2618157770 2023-11-04T05:28:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f262067f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26180c7130 con 0x7f261815de70 2023-11-04T05:28:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f262067f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26180c7410 con 0x7f2618157b50 2023-11-04T05:28:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.833+0000 7f57eb7fe700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec0c9b40 0x7f57ec14bae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57e400caa0 tx=0x7f57e400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fc1ed2e7700 1 Processor -- start 2023-11-04T05:28:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f4202690700 1 Processor -- start 2023-11-04T05:28:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261ec1c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 0x7f26180c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2618157b50 0x7f26180c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180004450 con 0x7f319006eeb0 2023-11-04T05:28:47.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31800045b0 con 0x7f319006eeb0 2023-11-04T05:28:47.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800038a0 con 0x7f319006eeb0 2023-11-04T05:28:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2618157b50 0x7f26180c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43086/0 (socket says 172.21.15.3:43086) 2023-11-04T05:28:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 -- 172.21.15.3:0/2897016630 learned_addr learned my addr 172.21.15.3:0/2897016630 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261e41b700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618157770 0x7f26180b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/2102848066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd4006170 msgr2=0x7fdbd4006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fdbf2dbf700 1 --2- 172.21.15.3:0/2102848066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd4006170 0x7fdbd4006a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009800 tx=0x7fdbd4007a90 comp rx=0 tx=0).stop 2023-11-04T05:28:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fdbf135c700 1 -- 172.21.15.3:0/2102848066 reap_dead start 2023-11-04T05:28:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/2102848066 shutdown_connections 2023-11-04T05:28:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/2102848066 >> 172.21.15.3:0/2102848066 conn(0x7fdbec0b1820 msgr2=0x7fdbec0b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 -- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 msgr2=0x7f26180c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f261815de70 0x7f26180c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 -- 172.21.15.3:0/2897016630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618157770 msgr2=0x7f26180b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618157770 0x7f26180b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 -- 172.21.15.3:0/2897016630 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608009550 con 0x7f2618157b50 2023-11-04T05:28:47.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f261dc1a700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2618157b50 0x7f26180c81d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f261400b520 tx=0x7f261400b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7fc1ed2e7700 1 -- start start 2023-11-04T05:28:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1ed2e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1e81579a0 0x7fc1e80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1ed2e7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e815d930 0x7fc1e815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1ed2e7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e815e2d0 con 0x7fc1e815a340 2023-11-04T05:28:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1ed2e7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e8157d80 con 0x7fc1e81579a0 2023-11-04T05:28:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1ed2e7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e80729a0 con 0x7fc1e815d930 2023-11-04T05:28:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1e6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1e81579a0 0x7fc1e80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1e6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1e81579a0 0x7fc1e80cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52034/0 (socket says 172.21.15.3:52034) 2023-11-04T05:28:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/2574813714 learned_addr learned my addr 172.21.15.3:0/2574813714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/2574813714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e815d930 msgr2=0x7fc1e815dd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fc1e6ffd700 1 --2- 172.21.15.3:0/2574813714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e815d930 0x7fc1e815dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/2574813714 >> v1:172.21.15.188:6789/0 conn(0x7fc1e815a340 legacy=0x7fc1e815a7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400a880 con 0x7f57ec0c9b40 2023-11-04T05:28:47.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e400a9e0 con 0x7f57ec0c9b40 2023-11-04T05:28:47.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400ad50 con 0x7f57ec0c9b40 2023-11-04T05:28:47.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31900c43c0 con 0x7f319006eeb0 2023-11-04T05:28:47.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31900c4980 con 0x7f319006eeb0 2023-11-04T05:28:47.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.834+0000 7f4202690700 1 -- start start 2023-11-04T05:28:47.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f4202690700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc1579a0 0x7f41fc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f4202690700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41fc15d930 0x7f41fc15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f4202690700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc157d80 con 0x7f41fc15a340 2023-11-04T05:28:47.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f4202690700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc15e2f0 con 0x7f41fc15d930 2023-11-04T05:28:47.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f4202690700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0729a0 con 0x7f41fc1579a0 2023-11-04T05:28:47.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f41fb7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41fc15d930 0x7f41fc15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f41fb7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41fc15d930 0x7f41fc15ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51120/0 (socket says 172.21.15.3:51120) 2023-11-04T05:28:47.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1785980533 learned_addr learned my addr 172.21.15.3:0/1785980533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f41fbfff700 1 --2- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc1579a0 0x7f41fc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc1579a0 msgr2=0x7f41fc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41fb7fe700 1 --2- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc1579a0 0x7f41fc0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1785980533 >> v1:172.21.15.188:6789/0 conn(0x7f41fc15a340 legacy=0x7f41fc15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1785980533 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc071900 con 0x7f41fc15d930 2023-11-04T05:28:47.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/2574813714 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8071900 con 0x7fc1e81579a0 2023-11-04T05:28:47.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0af8a0 con 0x7f57ec0c9b40 2023-11-04T05:28:47.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.835+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec0afe60 con 0x7f57ec0c9b40 2023-11-04T05:28:47.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57cc000fc0 con 0x7f57ec0c9b40 2023-11-04T05:28:47.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41fb7fe700 1 --2- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc15d930 0x7f41fc15ddb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098f0 tx=0x7f41f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06741e1b4922168 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fc1e6ffd700 1 --2- 172.21.15.3:0/2574813714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e81579a0 0x7fc1e80cc760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000d370 tx=0x7fc1e000d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3498a008ce0396da server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41faffd700 1 -- 172.21.15.3:0/1785980533 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000b8a0 con 0x7f41fc15d930 2023-11-04T05:28:47.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/2102848066 shutdown_connections 2023-11-04T05:28:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7fc1e5ffb700 1 -- 172.21.15.3:0/2574813714 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0010070 con 0x7fc1e81579a0 2023-11-04T05:28:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7fc1e5ffb700 1 -- 172.21.15.3:0/2574813714 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e000dcd0 con 0x7fc1e81579a0 2023-11-04T05:28:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7fc1e5ffb700 1 -- 172.21.15.3:0/2574813714 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0007d70 con 0x7fc1e81579a0 2023-11-04T05:28:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614011830 con 0x7f2618157b50 2023-11-04T05:28:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614011990 con 0x7f2618157b50 2023-11-04T05:28:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614011d00 con 0x7f2618157b50 2023-11-04T05:28:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26180c76f0 con 0x7f2618157b50 2023-11-04T05:28:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26181f6f20 con 0x7f2618157b50 2023-11-04T05:28:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3190072d50 con 0x7f319006eeb0 2023-11-04T05:28:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3180021070 con 0x7f319006eeb0 2023-11-04T05:28:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f31867fc700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f317406afd0 0x7f317406d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/2102848066 wait complete. 2023-11-04T05:28:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f318008aa40 con 0x7f319006eeb0 2023-11-04T05:28:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41faffd700 1 -- 172.21.15.3:0/1785980533 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f000ba00 con 0x7f41fc15d930 2023-11-04T05:28:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.836+0000 7f41faffd700 1 -- 172.21.15.3:0/1785980533 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000bd90 con 0x7f41fc15d930 2023-11-04T05:28:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f4202690700 1 -- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc15d930 msgr2=0x7f41fc15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.837+0000 7f4202690700 1 --2- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc15d930 0x7f41fc15ddb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098f0 tx=0x7f41f0009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7fdbf2dbf700 1 Processor -- start 2023-11-04T05:28:47.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57e4006070 con 0x7f57ec0c9b40 2023-11-04T05:28:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/2574813714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e81579a0 msgr2=0x7fc1e80cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/2574813714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e81579a0 0x7fc1e80cc760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000d370 tx=0x7fc1e000d680 comp rx=0 tx=0).stop 2023-11-04T05:28:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/2574813714 shutdown_connections 2023-11-04T05:28:47.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/2574813714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e81579a0 0x7fc1e80cc760 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/2574813714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e815d930 0x7fc1e815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/2574813714 >> 172.21.15.3:0/2574813714 conn(0x7fc1e806b150 msgr2=0x7fc1e806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f4202690700 1 -- 172.21.15.3:0/1785980533 shutdown_connections 2023-11-04T05:28:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f4202690700 1 --2- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc1579a0 0x7f41fc0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f4202690700 1 --2- 172.21.15.3:0/1785980533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc15d930 0x7f41fc15ddb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.838+0000 7f4202690700 1 -- 172.21.15.3:0/1785980533 >> 172.21.15.3:0/1785980533 conn(0x7f41fc06b150 msgr2=0x7f41fc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f4202690700 1 -- 172.21.15.3:0/1785980533 shutdown_connections 2023-11-04T05:28:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f57d3fff700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f57d806af50 0x7f57d806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f57e4089e10 con 0x7f57ec0c9b40 2023-11-04T05:28:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f57d3fff700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] conn(0x7f57d8074590 0x7f57d80769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 --> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57d80770a0 con 0x7f57d8074590 2023-11-04T05:28:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e408a2b0 con 0x7f57ec0c9b40 2023-11-04T05:28:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f3195528700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f317406afd0 0x7f317406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f31867fc700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] conn(0x7f3174074590 0x7f31740769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 --> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31740770a0 con 0x7f3174074590 2023-11-04T05:28:47.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7fdbf2dbf700 1 -- start start 2023-11-04T05:28:47.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31800556a0 con 0x7f319006eeb0 2023-11-04T05:28:47.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f261401a400 con 0x7f2618157b50 2023-11-04T05:28:47.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f260effd700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25fc06afd0 0x7f25fc06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2614089800 con 0x7f2618157b50 2023-11-04T05:28:47.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f57eaffd700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f57d806af50 0x7f57d806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.839+0000 7f4202690700 1 -- 172.21.15.3:0/1785980533 wait complete. 2023-11-04T05:28:47.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf2dbf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbec14f6f0 0x7fdbec158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/2574813714 shutdown_connections 2023-11-04T05:28:47.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/2574813714 wait complete. 2023-11-04T05:28:47.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf2dbf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbec14fd80 0x7fdbec150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf2dbf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0c8190 0x7fdbec0c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf2dbf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c8b80 con 0x7fdbec14f6f0 2023-11-04T05:28:47.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf2dbf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0b1550 con 0x7fdbec14fd80 2023-11-04T05:28:47.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf2dbf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec158740 con 0x7fdbec0c8190 2023-11-04T05:28:47.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf0b5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbec14f6f0 0x7fdbec158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf0b5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbec14f6f0 0x7fdbec158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52050/0 (socket says 172.21.15.3:52050) 2023-11-04T05:28:47.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf0b5b700 1 -- 172.21.15.3:0/1843953986 learned_addr learned my addr 172.21.15.3:0/1843953986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7fdbf135c700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0c8190 0x7fdbec0c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f57ebfff700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] conn(0x7f57d8074590 0x7f57d80769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7f57ebfff700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] conn(0x7f57d8074590 0x7f57d80769d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f261e41b700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25fc06afd0 0x7f25fc06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f262067f700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] conn(0x7f26000015e0 0x7f2600003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 --> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2600006bd0 con 0x7f26000015e0 2023-11-04T05:28:47.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7fdbebfff700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbec14fd80 0x7fdbec150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.840+0000 7f4202690700 1 Processor -- start 2023-11-04T05:28:47.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7f261ec1c700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] conn(0x7f26000015e0 0x7f2600003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7f57eaffd700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f57d806af50 0x7f57d806d410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f57e00090b0 tx=0x7f57e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7f319652a700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] conn(0x7f3174074590 0x7f31740769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7f319652a700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] conn(0x7f3174074590 0x7f31740769d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.841+0000 7f4202690700 1 -- start start 2023-11-04T05:28:47.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f3195528700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f317406afd0 0x7f317406d490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f318c009ad0 tx=0x7f318c009290 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== osd.2 v2:172.21.15.3:6800/3907251966 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f57d80770a0 con 0x7f57d8074590 2023-11-04T05:28:47.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fc1ed2e7700 1 Processor -- start 2023-11-04T05:28:47.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fdbf135c700 1 -- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbec14fd80 msgr2=0x7fdbec150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fdbf135c700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbec14fd80 0x7fdbec150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fdbf135c700 1 -- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbec14f6f0 msgr2=0x7fdbec158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fdbf135c700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbec14f6f0 0x7fdbec158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fc1ed2e7700 1 -- start start 2023-11-04T05:28:47.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7fdbf135c700 1 -- 172.21.15.3:0/1843953986 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec157160 con 0x7fdbec0c8190 2023-11-04T05:28:47.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f4202690700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc1579a0 0x7f41fc0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f4202690700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41fc15a340 0x7f41fc0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f4202690700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc15d930 0x7f41fc0c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f4202690700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0c7e00 con 0x7f41fc15a340 2023-11-04T05:28:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f4202690700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0c80e0 con 0x7f41fc1579a0 2023-11-04T05:28:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f4202690700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0c83c0 con 0x7f41fc15d930 2023-11-04T05:28:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f561e7e5700 1 Processor -- start 2023-11-04T05:28:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f261e41b700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25fc06afd0 0x7f25fc06d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f261000a510 tx=0x7f261000d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== osd.3 v2:172.21.15.3:6817/4243967239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f31740770a0 con 0x7f3174074590 2023-11-04T05:28:47.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.842+0000 7f561e7e5700 1 -- start start 2023-11-04T05:28:47.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f561e7e5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f561815d930 0x7f561815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f561e7e5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f56180c9b40 0x7f56180c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f561e7e5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5618157d80 con 0x7f56181579a0 2023-11-04T05:28:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f561e7e5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561815e300 con 0x7f56180c9b40 2023-11-04T05:28:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f561e7e5700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56180729a0 con 0x7f561815d930 2023-11-04T05:28:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7fc1ed2e7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e81579a0 0x7fc1e8155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1ed2e7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1e815a340 0x7fc1e80b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1ed2e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e815d930 0x7fc1e80a8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1ed2e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e81561c0 con 0x7fc1e815d930 2023-11-04T05:28:47.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1ed2e7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e80b1c80 con 0x7fc1e815a340 2023-11-04T05:28:47.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1ed2e7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e80b19a0 con 0x7fc1e81579a0 2023-11-04T05:28:47.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e81579a0 0x7fc1e8155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e81579a0 0x7fc1e8155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43124/0 (socket says 172.21.15.3:43124) 2023-11-04T05:28:47.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/3109447859 learned_addr learned my addr 172.21.15.3:0/3109447859 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e67fc700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1e815a340 0x7fc1e80b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1e815a340 msgr2=0x7fc1e80b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1e815a340 0x7fc1e80b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e815d930 msgr2=0x7fc1e80a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1e815d930 0x7fc1e80a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7fc1e6ffd700 1 -- 172.21.15.3:0/3109447859 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e000d050 con 0x7fc1e81579a0 2023-11-04T05:28:47.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7fc1e6ffd700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e81579a0 0x7fc1e8155c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1e806dc90 tx=0x7fc1e0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f261ec1c700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] conn(0x7f26000015e0 0x7f2600003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f5617fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56181579a0 0x7f56180cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57412/0 (socket says 172.21.15.3:57412) 2023-11-04T05:28:47.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f5617fff700 1 -- 172.21.15.3:0/167194501 learned_addr learned my addr 172.21.15.3:0/167194501 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:47.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f561cd82700 1 --2- 172.21.15.3:0/167194501 >> v2:172.21.15.3:3300/0 conn(0x7f56180c9b40 0x7f56180c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f56177fe700 1 --2- 172.21.15.3:0/167194501 >> v2:172.21.15.188:3300/0 conn(0x7f561815d930 0x7f561815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f561cd82700 1 -- 172.21.15.3:0/167194501 >> v2:172.21.15.188:3300/0 conn(0x7f561815d930 msgr2=0x7f561815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f561cd82700 1 --2- 172.21.15.3:0/167194501 >> v2:172.21.15.188:3300/0 conn(0x7f561815d930 0x7f561815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f561cd82700 1 -- 172.21.15.3:0/167194501 >> v1:172.21.15.3:6789/0 conn(0x7f56181579a0 legacy=0x7f56180cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f561cd82700 1 -- 172.21.15.3:0/167194501 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618071900 con 0x7f56180c9b40 2023-11-04T05:28:47.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7f561cd82700 1 --2- 172.21.15.3:0/167194501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56180c9b40 0x7f56180c9f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f56180cd4b0 tx=0x7f561000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f336f31c6c4da75a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== osd.5 v2:172.21.15.188:6814/3601495681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f2600006bd0 con 0x7f26000015e0 2023-11-04T05:28:47.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7f5616ffd700 1 -- 172.21.15.3:0/167194501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000f070 con 0x7f56180c9b40 2023-11-04T05:28:47.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7f5616ffd700 1 -- 172.21.15.3:0/167194501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561000a2e0 con 0x7f56180c9b40 2023-11-04T05:28:47.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7f5616ffd700 1 -- 172.21.15.3:0/167194501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000cbe0 con 0x7f56180c9b40 2023-11-04T05:28:47.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7fdbf135c700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0c8190 0x7fdbec0c8640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbec06d160 tx=0x7fdbdc00c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0115f0 con 0x7fdbec0c8190 2023-11-04T05:28:47.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.845+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc011750 con 0x7fdbec0c8190 2023-11-04T05:28:47.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc011b00 con 0x7fdbec0c8190 2023-11-04T05:28:47.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec157440 con 0x7fdbec0c8190 2023-11-04T05:28:47.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec157a00 con 0x7fdbec0c8190 2023-11-04T05:28:47.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0004690 con 0x7fc1e81579a0 2023-11-04T05:28:47.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e00047f0 con 0x7fc1e81579a0 2023-11-04T05:28:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0008230 con 0x7fc1e81579a0 2023-11-04T05:28:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e81e3d00 con 0x7fc1e81579a0 2023-11-04T05:28:47.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e81e42c0 con 0x7fc1e81579a0 2023-11-04T05:28:47.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7f561e7e5700 1 -- 172.21.15.3:0/167194501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56180c9b40 msgr2=0x7f56180c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.846+0000 7f561e7e5700 1 --2- 172.21.15.3:0/167194501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56180c9b40 0x7f56180c9f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f56180cd4b0 tx=0x7f561000c330 comp rx=0 tx=0).stop 2023-11-04T05:28:47.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.847+0000 7f561e7e5700 1 -- 172.21.15.3:0/167194501 shutdown_connections 2023-11-04T05:28:47.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.847+0000 7f561e7e5700 1 --2- 172.21.15.3:0/167194501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56180c9b40 0x7f56180c9f60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.847+0000 7f561e7e5700 1 --2- 172.21.15.3:0/167194501 >> v2:172.21.15.188:3300/0 conn(0x7f561815d930 0x7f561815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.847+0000 7f561e7e5700 1 -- 172.21.15.3:0/167194501 >> 172.21.15.3:0/167194501 conn(0x7f561806b150 msgr2=0x7f561806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.847+0000 7f561e7e5700 1 -- 172.21.15.3:0/167194501 shutdown_connections 2023-11-04T05:28:47.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.847+0000 7f561e7e5700 1 -- 172.21.15.3:0/167194501 wait complete. 2023-11-04T05:28:47.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41fc15a340 0x7f41fc0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41fc15a340 0x7f41fc0b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52058/0 (socket says 172.21.15.3:52058) 2023-11-04T05:28:47.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1155110501 learned_addr learned my addr 172.21.15.3:0/1155110501 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc15d930 msgr2=0x7f41fc0c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc15d930 0x7f41fc0c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc1579a0 msgr2=0x7f41fc0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc1579a0 0x7f41fc0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.843+0000 7f41fb7fe700 1 -- 172.21.15.3:0/1155110501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009550 con 0x7f41fc15a340 2023-11-04T05:28:47.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f41fb7fe700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41fc15a340 0x7f41fc0b14c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009c40 tx=0x7f41f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004510 con 0x7f41fc15a340 2023-11-04T05:28:47.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f001f070 con 0x7f41fc15a340 2023-11-04T05:28:47.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.844+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00038c0 con 0x7f41fc15a340 2023-11-04T05:28:47.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.848+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdbdc011c60 con 0x7fdbec0c8190 2023-11-04T05:28:47.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.848+0000 7fdbe97fa700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbd806af50 0x7fdbd806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.848+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc0c86a0 con 0x7f41fc15a340 2023-11-04T05:28:47.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.848+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc0ae7b0 con 0x7f41fc15a340 2023-11-04T05:28:47.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.849+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbdc08e060 con 0x7fdbec0c8190 2023-11-04T05:28:47.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.849+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41fc0c8250 con 0x7f41fc15a340 2023-11-04T05:28:47.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.849+0000 7fdbf0b5b700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbd806af50 0x7fdbd806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.849+0000 7f561e7e5700 1 Processor -- start 2023-11-04T05:28:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.849+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc1e0008390 con 0x7fc1e81579a0 2023-11-04T05:28:47.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.850+0000 7fdbf0b5b700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbd806af50 0x7fdbd806d410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001ba0 tx=0x7fdbe4001e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.849+0000 7fc1cf7fe700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc1d406afd0 0x7fc1d406d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.850+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc1e000de40 con 0x7fc1e81579a0 2023-11-04T05:28:47.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.850+0000 7fc1e67fc700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc1d406afd0 0x7fc1d406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7fc1e67fc700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc1d406afd0 0x7fc1d406d490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80c5790 tx=0x7fc1d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] conn(0x7fc1c80015e0 0x7fc1c8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 --> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1c8006bd0 con 0x7fc1c80015e0 2023-11-04T05:28:47.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7fdbf2dbf700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] conn(0x7fdbec152210 0x7fdbec1e1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 --> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbec0cb430 con 0x7fdbec152210 2023-11-04T05:28:47.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7efd31b32700 1 Processor -- start 2023-11-04T05:28:47.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.850+0000 7f561e7e5700 1 -- start start 2023-11-04T05:28:47.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f561e7e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56180c9b40 0x7f56181f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f561e7e5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56181579a0 0x7f56180c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f561e7e5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f561815d930 0x7f56180c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f561e7e5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180afd90 con 0x7f56181579a0 2023-11-04T05:28:47.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f561e7e5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180b0070 con 0x7f56180c9b40 2023-11-04T05:28:47.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f561e7e5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56180b0350 con 0x7f561815d930 2023-11-04T05:28:47.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41f0003a20 con 0x7f41fc15a340 2023-11-04T05:28:47.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7efd31b32700 1 -- start start 2023-11-04T05:28:47.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f5617fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56180c9b40 0x7f56181f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f56177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56181579a0 0x7f56180c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f56177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56181579a0 0x7f56180c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52082/0 (socket says 172.21.15.3:52082) 2023-11-04T05:28:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f56177fe700 1 -- 172.21.15.3:0/1598672680 learned_addr learned my addr 172.21.15.3:0/1598672680 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7fdbebfff700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] conn(0x7fdbec152210 0x7fdbec1e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.851+0000 7f41f8ff9700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f41e806afd0 0x7f41e806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f41f0021070 con 0x7f41fc15a340 2023-11-04T05:28:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f41f8ff9700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] conn(0x7f41e8074610 0x7f41e8076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 --> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41e8077120 con 0x7f41e8074610 2023-11-04T05:28:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f008bad0 con 0x7f41fc15a340 2023-11-04T05:28:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7fdbebfff700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] conn(0x7fdbec152210 0x7fdbec1e1640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f41fbfff700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f41e806afd0 0x7f41e806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5617fff700 1 -- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f561815d930 msgr2=0x7f56180c25c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-04T05:28:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5617fff700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f561815d930 0x7f56180c25c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5617fff700 1 -- 172.21.15.3:0/1598672680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56181579a0 msgr2=0x7f56180c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5617fff700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56181579a0 0x7f56180c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5617fff700 1 -- 172.21.15.3:0/1598672680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561000ab80 con 0x7f56180c9b40 2023-11-04T05:28:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5617fff700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56180c9b40 0x7f56181f62f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f560800c930 tx=0x7f560800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608007970 con 0x7f56180c9b40 2023-11-04T05:28:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608007ad0 con 0x7f56180c9b40 2023-11-04T05:28:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608007e40 con 0x7f56180c9b40 2023-11-04T05:28:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56180b0630 con 0x7f56180c9b40 2023-11-04T05:28:47.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31b32700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd2403a6b0 0x7efd240386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31b32700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd24033610 0x7efd24050000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31b32700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd24033e40 con 0x7efd240e0f60 2023-11-04T05:28:47.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31b32700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2404f820 con 0x7efd2403a6b0 2023-11-04T05:28:47.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31b32700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd24033b60 con 0x7efd24033610 2023-11-04T05:28:47.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd30b30700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efd240e0f60 0x7efd24039fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57440/0 (socket says 172.21.15.3:57440) 2023-11-04T05:28:47.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd30b30700 1 -- 172.21.15.3:0/3328416802 learned_addr learned my addr 172.21.15.3:0/3328416802 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:47.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.852+0000 7f4200c2d700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] conn(0x7f41e8074610 0x7f41e8076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31331700 1 --2- 172.21.15.3:0/3328416802 >> v2:172.21.15.188:3300/0 conn(0x7efd24033610 0x7efd24050000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd2bfff700 1 --2- 172.21.15.3:0/3328416802 >> v2:172.21.15.3:3300/0 conn(0x7efd2403a6b0 0x7efd240386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31331700 1 -- 172.21.15.3:0/3328416802 >> v2:172.21.15.3:3300/0 conn(0x7efd2403a6b0 msgr2=0x7efd240386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31331700 1 --2- 172.21.15.3:0/3328416802 >> v2:172.21.15.3:3300/0 conn(0x7efd2403a6b0 0x7efd240386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31331700 1 -- 172.21.15.3:0/3328416802 >> v1:172.21.15.3:6789/0 conn(0x7efd240e0f60 legacy=0x7efd24039fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7efd31331700 1 -- 172.21.15.3:0/3328416802 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2404eb80 con 0x7efd24033610 2023-11-04T05:28:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7efd31331700 1 --2- 172.21.15.3:0/3328416802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24033610 0x7efd24050000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd2c069f70 tx=0x7efd2c04ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29553faa9270fb server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7f4200c2d700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] conn(0x7f41e8074610 0x7f41e8076a50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.853+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181f6830 con 0x7f56180c9b40 2023-11-04T05:28:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7f41fbfff700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f41e806afd0 0x7f41e806d490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00afd0 tx=0x7f41ec00c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== osd.0 v2:172.21.15.3:6816/2508882777 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fdbe0004210 con 0x7fdbec152210 2023-11-04T05:28:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7fc1e77fe700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] conn(0x7fc1c80015e0 0x7fc1c8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5618071ec0 con 0x7f56180c9b40 2023-11-04T05:28:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7fc1e77fe700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] conn(0x7fc1c80015e0 0x7fc1c8003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7efd2b7fe700 1 -- 172.21.15.3:0/3328416802 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c08e070 con 0x7efd24033610 2023-11-04T05:28:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7efd2b7fe700 1 -- 172.21.15.3:0/3328416802 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c0731c0 con 0x7efd24033610 2023-11-04T05:28:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.854+0000 7efd2b7fe700 1 -- 172.21.15.3:0/3328416802 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c067bb0 con 0x7efd24033610 2023-11-04T05:28:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.855+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== osd.4 v2:172.21.15.188:6800/989266714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f41e8077120 con 0x7f41e8074610 2023-11-04T05:28:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 -- 172.21.15.3:0/3328416802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24033610 msgr2=0x7efd24050000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 --2- 172.21.15.3:0/3328416802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24033610 0x7efd24050000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd2c069f70 tx=0x7efd2c04ffa0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 -- 172.21.15.3:0/3328416802 shutdown_connections 2023-11-04T05:28:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 --2- 172.21.15.3:0/3328416802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd24033610 0x7efd24050000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 --2- 172.21.15.3:0/3328416802 >> v2:172.21.15.3:3300/0 conn(0x7efd2403a6b0 0x7efd240386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 -- 172.21.15.3:0/3328416802 >> 172.21.15.3:0/3328416802 conn(0x7efd24054b30 msgr2=0x7efd24054f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:47.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 -- 172.21.15.3:0/3328416802 shutdown_connections 2023-11-04T05:28:47.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.856+0000 7efd31b32700 1 -- 172.21.15.3:0/3328416802 wait complete. 2023-11-04T05:28:47.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.858+0000 7efd31b32700 1 Processor -- start 2023-11-04T05:28:47.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.858+0000 7efd31b32700 1 -- start start 2023-11-04T05:28:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.859+0000 7efd31b32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd24033610 0x7efd240d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31b32700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd2403a6b0 0x7efd24036520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31b32700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240e0f60 0x7efd2404d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31b32700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd240d51e0 con 0x7efd24033610 2023-11-04T05:28:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31b32700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24048980 con 0x7efd2403a6b0 2023-11-04T05:28:47.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31b32700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd24048c60 con 0x7efd240e0f60 2023-11-04T05:28:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5608016420 con 0x7f56180c9b40 2023-11-04T05:28:47.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31331700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240e0f60 0x7efd2404d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31331700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240e0f60 0x7efd2404d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43156/0 (socket says 172.21.15.3:43156) 2023-11-04T05:28:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7efd31331700 1 -- 172.21.15.3:0/1734643133 learned_addr learned my addr 172.21.15.3:0/1734643133 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7efd31331700 1 -- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd2403a6b0 msgr2=0x7efd24036520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7efd31331700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd2403a6b0 0x7efd24036520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7efd31331700 1 -- 172.21.15.3:0/1734643133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd24033610 msgr2=0x7efd240d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7efd31331700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd24033610 0x7efd240d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:47.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7efd31331700 1 -- 172.21.15.3:0/1734643133 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24048f40 con 0x7efd240e0f60 2023-11-04T05:28:47.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.860+0000 7f5614ff9700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f560406af00 0x7f560406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5608012070 con 0x7f56180c9b40 2023-11-04T05:28:47.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f56177fe700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f560406af00 0x7f560406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f5614ff9700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] conn(0x7f5604074540 0x7f5604076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:47.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 --> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5604077050 con 0x7f5604074540 2023-11-04T05:28:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f560808a8c0 con 0x7f56180c9b40 2023-11-04T05:28:47.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7efd31331700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240e0f60 0x7efd2404d230 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd2c068a80 tx=0x7efd2c06a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f561cd82700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] conn(0x7f5604074540 0x7f5604076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.861+0000 7f56177fe700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f560406af00 0x7f560406d3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56181582c0 tx=0x7f560c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.862+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c097040 con 0x7efd240e0f60 2023-11-04T05:28:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.862+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c0738a0 con 0x7efd240e0f60 2023-11-04T05:28:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.862+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c073c10 con 0x7efd240e0f60 2023-11-04T05:28:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.862+0000 7f561cd82700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] conn(0x7f5604074540 0x7f5604076980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:47.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.862+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== osd.7 v2:172.21.15.188:6824/1844078319 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f5604077050 con 0x7f5604074540 2023-11-04T05:28:48.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.863+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd24049220 con 0x7efd240e0f60 2023-11-04T05:28:48.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.863+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd24048630 con 0x7efd240e0f60 2023-11-04T05:28:48.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.865+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd2c073d70 con 0x7efd240e0f60 2023-11-04T05:28:48.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.865+0000 7efd297fa700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7efd1006af50 0x7efd1006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.865+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efd2c104ac0 con 0x7efd240e0f60 2023-11-04T05:28:48.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.865+0000 7efd30b30700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7efd1006af50 0x7efd1006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.866+0000 7efd30b30700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7efd1006af50 0x7efd1006d410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd20009730 tx=0x7efd20006c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.867+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== osd.1 v2:172.21.15.3:6802/2297993387 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fc1c8006bd0 con 0x7fc1c80015e0 2023-11-04T05:28:48.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.869+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 --> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3190067f10 con 0x7f3174074590 2023-11-04T05:28:48.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.873+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 --> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57cc002d40 con 0x7f57d8074590 2023-11-04T05:28:48.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.874+0000 7f57d3fff700 1 -- 172.21.15.3:0/479088342 <== osd.2 v2:172.21.15.3:6800/3907251966 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57cc002d40 con 0x7f57d8074590 2023-11-04T05:28:48.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.874+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] conn(0x7f57d8074590 msgr2=0x7f57d80769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.874+0000 7f57f1ecc700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:6800/3907251966,v1:172.21.15.3:6801/3907251966] conn(0x7f57d8074590 0x7f57d80769d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.876+0000 7f31867fc700 1 -- 172.21.15.3:0/2577430771 <== osd.3 v2:172.21.15.3:6817/4243967239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3190067f10 con 0x7f3174074590 2023-11-04T05:28:48.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.876+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f57d806af50 msgr2=0x7f57d806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.876+0000 7f57f1ecc700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f57d806af50 0x7f57d806d410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f57e00090b0 tx=0x7f57e0008030 comp rx=0 tx=0).stop 2023-11-04T05:28:48.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.876+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec0c9b40 msgr2=0x7f57ec14bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.876+0000 7f57f1ecc700 1 --2- 172.21.15.3:0/479088342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec0c9b40 0x7f57ec14bae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57e400caa0 tx=0x7f57e400ce60 comp rx=0 tx=0).stop 2023-11-04T05:28:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.876+0000 7f57ebfff700 1 -- 172.21.15.3:0/479088342 reap_dead start 2023-11-04T05:28:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.877+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 --> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbec15a5e0 con 0x7fdbec152210 2023-11-04T05:28:48.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbe97fa700 1 -- 172.21.15.3:0/1843953986 <== osd.0 v2:172.21.15.3:6816/2508882777 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdbe0004380 con 0x7fdbec152210 2023-11-04T05:28:48.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] conn(0x7fdbec152210 msgr2=0x7fdbec1e1640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.3:6816/2508882777,v1:172.21.15.3:6818/2508882777] conn(0x7fdbec152210 0x7fdbec1e1640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbd806af50 msgr2=0x7fdbd806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbd806af50 0x7fdbd806d410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001ba0 tx=0x7fdbe4001e90 comp rx=0 tx=0).stop 2023-11-04T05:28:48.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0c8190 msgr2=0x7fdbec0c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 --2- 172.21.15.3:0/1843953986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbec0c8190 0x7fdbec0c8640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbec06d160 tx=0x7fdbdc00c660 comp rx=0 tx=0).stop 2023-11-04T05:28:48.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf135c700 1 -- 172.21.15.3:0/1843953986 reap_dead start 2023-11-04T05:28:48.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7efd31b32700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] conn(0x7efd24049a20 0x7efd240060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 --> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd2404ee60 con 0x7efd24049a20 2023-11-04T05:28:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 shutdown_connections 2023-11-04T05:28:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 >> 172.21.15.3:0/1843953986 conn(0x7fdbec0b1820 msgr2=0x7fdbec0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.878+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 --> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2600005c50 con 0x7f26000015e0 2023-11-04T05:28:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 shutdown_connections 2023-11-04T05:28:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7fdbf2dbf700 1 -- 172.21.15.3:0/1843953986 wait complete. 2023-11-04T05:28:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7efd2bfff700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] conn(0x7efd24049a20 0x7efd240060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7f260effd700 1 -- 172.21.15.3:0/2897016630 <== osd.5 v2:172.21.15.188:6814/3601495681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2600005c50 con 0x7f26000015e0 2023-11-04T05:28:48.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7efd2bfff700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] conn(0x7efd24049a20 0x7efd240060f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] conn(0x7f26000015e0 msgr2=0x7f2600003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.879+0000 7f262067f700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6814/3601495681,v1:172.21.15.188:6816/3601495681] conn(0x7f26000015e0 0x7f2600003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== osd.6 v2:172.21.15.188:6808/830502099 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7efd18009480 con 0x7efd24049a20 2023-11-04T05:28:48.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] conn(0x7f3174074590 msgr2=0x7f31740769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f3197f8d700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.3:6817/4243967239,v1:172.21.15.3:6819/4243967239] conn(0x7f3174074590 0x7f31740769d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f317406afd0 msgr2=0x7f317406d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f3197f8d700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f317406afd0 0x7f317406d490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f318c009ad0 tx=0x7f318c009290 comp rx=0 tx=0).stop 2023-11-04T05:28:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319006eeb0 msgr2=0x7f31900c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f3197f8d700 1 --2- 172.21.15.3:0/2577430771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319006eeb0 0x7f31900c3a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3180016000 tx=0x7f3180004750 comp rx=0 tx=0).stop 2023-11-04T05:28:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.881+0000 7f319652a700 1 -- 172.21.15.3:0/2577430771 reap_dead start 2023-11-04T05:28:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.887+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 shutdown_connections 2023-11-04T05:28:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.887+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 >> 172.21.15.3:0/479088342 conn(0x7f57ec06b150 msgr2=0x7f57ec0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.887+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 --> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5618071be0 con 0x7f5604074540 2023-11-04T05:28:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.887+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 shutdown_connections 2023-11-04T05:28:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.887+0000 7f57f1ecc700 1 -- 172.21.15.3:0/479088342 wait complete. 2023-11-04T05:28:48.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.887+0000 7f5614ff9700 1 -- 172.21.15.3:0/1598672680 <== osd.7 v2:172.21.15.188:6824/1844078319 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f561000d9d0 con 0x7f5604074540 2023-11-04T05:28:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.889+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 shutdown_connections 2023-11-04T05:28:48.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.889+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 >> 172.21.15.3:0/2577430771 conn(0x7f319006b4d0 msgr2=0x7f319006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25fc06afd0 msgr2=0x7f25fc06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f25fc06afd0 0x7f25fc06d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f261000a510 tx=0x7f261000d040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2618157b50 msgr2=0x7f26180c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 --2- 172.21.15.3:0/2897016630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2618157b50 0x7f26180c81d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f261400b520 tx=0x7f261400b830 comp rx=0 tx=0).stop 2023-11-04T05:28:48.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f261ec1c700 1 -- 172.21.15.3:0/2897016630 reap_dead start 2023-11-04T05:28:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 shutdown_connections 2023-11-04T05:28:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 >> 172.21.15.3:0/2897016630 conn(0x7f261806b4d0 msgr2=0x7f261815b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 shutdown_connections 2023-11-04T05:28:48.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.880+0000 7f262067f700 1 -- 172.21.15.3:0/2897016630 wait complete. 2023-11-04T05:28:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.890+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 --> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41fc071ec0 con 0x7f41e8074610 2023-11-04T05:28:48.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.890+0000 7f41f8ff9700 1 -- 172.21.15.3:0/1155110501 <== osd.4 v2:172.21.15.188:6800/989266714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f41f4001d50 con 0x7f41e8074610 2023-11-04T05:28:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] conn(0x7f41e8074610 msgr2=0x7f41e8076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4202690700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6800/989266714,v1:172.21.15.188:6801/989266714] conn(0x7f41e8074610 0x7f41e8076a50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f41e806afd0 msgr2=0x7f41e806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4202690700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f41e806afd0 0x7f41e806d490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00afd0 tx=0x7f41ec00c040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41fc15a340 msgr2=0x7f41fc0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4202690700 1 --2- 172.21.15.3:0/1155110501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41fc15a340 0x7f41fc0b14c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009c40 tx=0x7f41f0004750 comp rx=0 tx=0).stop 2023-11-04T05:28:48.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.891+0000 7f4200c2d700 1 -- 172.21.15.3:0/1155110501 reap_dead start 2023-11-04T05:28:48.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] conn(0x7f5604074540 msgr2=0x7f5604076980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6824/1844078319,v1:172.21.15.188:6825/1844078319] conn(0x7f5604074540 0x7f5604076980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f560406af00 msgr2=0x7f560406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f560406af00 0x7f560406d3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56181582c0 tx=0x7f560c008040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56180c9b40 msgr2=0x7f56181f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 --2- 172.21.15.3:0/1598672680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56180c9b40 0x7f56181f62f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f560800c930 tx=0x7f560800cc40 comp rx=0 tx=0).stop 2023-11-04T05:28:48.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561cd82700 1 -- 172.21.15.3:0/1598672680 reap_dead start 2023-11-04T05:28:48.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 shutdown_connections 2023-11-04T05:28:48.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 >> 172.21.15.3:0/1598672680 conn(0x7f561806b150 msgr2=0x7f56180cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 shutdown_connections 2023-11-04T05:28:48.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.892+0000 7f561e7e5700 1 -- 172.21.15.3:0/1598672680 wait complete. 2023-11-04T05:28:48.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.893+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 shutdown_connections 2023-11-04T05:28:48.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.893+0000 7f3197f8d700 1 -- 172.21.15.3:0/2577430771 wait complete. 2023-11-04T05:28:48.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.895+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 --> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1c8005cb0 con 0x7fc1c80015e0 2023-11-04T05:28:48.028 INFO:teuthology.orchestra.run.smithi003.stdout:60129542155 2023-11-04T05:28:48.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-04T05:28:48.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.899+0000 7fc1cf7fe700 1 -- 172.21.15.3:0/3109447859 <== osd.1 v2:172.21.15.3:6802/2297993387 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1c8005cb0 con 0x7fc1c80015e0 2023-11-04T05:28:48.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.899+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] conn(0x7fc1c80015e0 msgr2=0x7fc1c8003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.899+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.3:6802/2297993387,v1:172.21.15.3:6804/2297993387] conn(0x7fc1c80015e0 0x7fc1c8003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.902+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 shutdown_connections 2023-11-04T05:28:48.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.902+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 >> 172.21.15.3:0/1155110501 conn(0x7f41fc06b150 msgr2=0x7f41fc15bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 shutdown_connections 2023-11-04T05:28:48.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc1d406afd0 msgr2=0x7fc1d406d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc1d406afd0 0x7fc1d406d490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80c5790 tx=0x7fc1d8008040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e81579a0 msgr2=0x7fc1e8155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1ed2e7700 1 --2- 172.21.15.3:0/3109447859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1e81579a0 0x7fc1e8155c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1e806dc90 tx=0x7fc1e0004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1e77fe700 1 -- 172.21.15.3:0/3109447859 reap_dead start 2023-11-04T05:28:48.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 shutdown_connections 2023-11-04T05:28:48.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.905+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 >> 172.21.15.3:0/3109447859 conn(0x7fc1e806b150 msgr2=0x7fc1e80ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.906+0000 7f4202690700 1 -- 172.21.15.3:0/1155110501 wait complete. 2023-11-04T05:28:48.033 INFO:teuthology.orchestra.run.smithi003.stdout:60129542155 2023-11-04T05:28:48.034 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-04T05:28:48.035 INFO:teuthology.orchestra.run.smithi003.stdout:64424509451 2023-11-04T05:28:48.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-04T05:28:48.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.912+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 --> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd24005e40 con 0x7efd24049a20 2023-11-04T05:28:48.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.912+0000 7efd297fa700 1 -- 172.21.15.3:0/1734643133 <== osd.6 v2:172.21.15.188:6808/830502099 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd18009880 con 0x7efd24049a20 2023-11-04T05:28:48.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.913+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 shutdown_connections 2023-11-04T05:28:48.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.913+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] conn(0x7efd24049a20 msgr2=0x7efd240060f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.913+0000 7efd31b32700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6808/830502099,v1:172.21.15.188:6809/830502099] conn(0x7efd24049a20 0x7efd240060f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.038 INFO:teuthology.orchestra.run.smithi003.stdout:64424509451 2023-11-04T05:28:48.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-04T05:28:48.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7fc1ed2e7700 1 -- 172.21.15.3:0/3109447859 wait complete. 2023-11-04T05:28:48.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7efd1006af50 msgr2=0x7efd1006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7efd1006af50 0x7efd1006d410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd20009730 tx=0x7efd20006c60 comp rx=0 tx=0).stop 2023-11-04T05:28:48.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240e0f60 msgr2=0x7efd2404d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 --2- 172.21.15.3:0/1734643133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240e0f60 0x7efd2404d230 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd2c068a80 tx=0x7efd2c06a410 comp rx=0 tx=0).stop 2023-11-04T05:28:48.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31331700 1 -- 172.21.15.3:0/1734643133 reap_dead start 2023-11-04T05:28:48.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 shutdown_connections 2023-11-04T05:28:48.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 >> 172.21.15.3:0/1734643133 conn(0x7efd24054b30 msgr2=0x7efd240df390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 shutdown_connections 2023-11-04T05:28:48.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:47.914+0000 7efd31b32700 1 -- 172.21.15.3:0/1734643133 wait complete. 2023-11-04T05:28:48.042 INFO:teuthology.orchestra.run.smithi003.stdout:60129542155 2023-11-04T05:28:48.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-04T05:28:48.046 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-11-04T05:28:48.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-04T05:28:48.048 INFO:teuthology.orchestra.run.smithi003.stdout:64424509452 2023-11-04T05:28:48.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-04T05:28:48.049 INFO:teuthology.orchestra.run.smithi003.stdout:60129542154 2023-11-04T05:28:48.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-04T05:28:48.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.281+0000 7ffbce039700 1 Processor -- start 2023-11-04T05:28:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.281+0000 7ffbce039700 1 -- start start 2023-11-04T05:28:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.282+0000 7ffbce039700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 0x7ffbc815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.282+0000 7ffbce039700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc815e2d0 con 0x7ffbc81579a0 2023-11-04T05:28:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbce039700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc8157d80 con 0x7ffbc80c9b40 2023-11-04T05:28:48.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbce039700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80729a0 con 0x7ffbc815d930 2023-11-04T05:28:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbc6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 0x7ffbc815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbc6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 0x7ffbc815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43170/0 (socket says 172.21.15.3:43170) 2023-11-04T05:28:48.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3159064428 learned_addr learned my addr 172.21.15.3:0/3159064428 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3159064428 >> v1:172.21.15.188:6789/0 conn(0x7ffbc80c9b40 legacy=0x7ffbc80c9fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-04T05:28:48.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3159064428 >> v1:172.21.15.3:6789/0 conn(0x7ffbc81579a0 legacy=0x7ffbc80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.283+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3159064428 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8071900 con 0x7ffbc815d930 2023-11-04T05:28:48.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.284+0000 7ffbc6ffd700 1 --2- 172.21.15.3:0/3159064428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 0x7ffbc815dd90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0007a60 tx=0x7ffbc0010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0539f6102f57f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.285+0000 7ffbc67fc700 1 -- 172.21.15.3:0/3159064428 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0010270 con 0x7ffbc815d930 2023-11-04T05:28:48.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.285+0000 7ffbc67fc700 1 -- 172.21.15.3:0/3159064428 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0019a60 con 0x7ffbc815d930 2023-11-04T05:28:48.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.285+0000 7ffbc67fc700 1 -- 172.21.15.3:0/3159064428 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0019e10 con 0x7ffbc815d930 2023-11-04T05:28:48.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.286+0000 7ffbce039700 1 -- 172.21.15.3:0/3159064428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 msgr2=0x7ffbc815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.286+0000 7ffbce039700 1 --2- 172.21.15.3:0/3159064428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 0x7ffbc815dd90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0007a60 tx=0x7ffbc0010830 comp rx=0 tx=0).stop 2023-11-04T05:28:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.289+0000 7ffbce039700 1 -- 172.21.15.3:0/3159064428 shutdown_connections 2023-11-04T05:28:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.289+0000 7ffbce039700 1 --2- 172.21.15.3:0/3159064428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc815d930 0x7ffbc815dd90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.289+0000 7ffbce039700 1 -- 172.21.15.3:0/3159064428 >> 172.21.15.3:0/3159064428 conn(0x7ffbc806b150 msgr2=0x7ffbc806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.292+0000 7ffbce039700 1 -- 172.21.15.3:0/3159064428 shutdown_connections 2023-11-04T05:28:48.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.292+0000 7ffbce039700 1 -- 172.21.15.3:0/3159064428 wait complete. 2023-11-04T05:28:48.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.294+0000 7ffbce039700 1 Processor -- start 2023-11-04T05:28:48.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.295+0000 7fb234f47700 1 Processor -- start 2023-11-04T05:28:48.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7f133aa6d700 1 Processor -- start 2023-11-04T05:28:48.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7f133aa6d700 1 -- start start 2023-11-04T05:28:48.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.295+0000 7ffbce039700 1 -- start start 2023-11-04T05:28:48.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7ffbce039700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc80c9b40 0x7ffbc80c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7ffbce039700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc81579a0 0x7ffbc80c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7ffbce039700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc815d930 0x7ffbc8153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7ffbce039700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80a8de0 con 0x7ffbc815d930 2023-11-04T05:28:48.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7ffbce039700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80a84d0 con 0x7ffbc81579a0 2023-11-04T05:28:48.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7ffbce039700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80c7ef0 con 0x7ffbc80c9b40 2023-11-04T05:28:48.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc81579a0 0x7ffbc80c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc815d930 0x7ffbc8153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc815d930 0x7ffbc8153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52122/0 (socket says 172.21.15.3:52122) 2023-11-04T05:28:48.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc7fff700 1 -- 172.21.15.3:0/3478340506 learned_addr learned my addr 172.21.15.3:0/3478340506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc80c9b40 msgr2=0x7ffbc80c8500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-04T05:28:48.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc6ffd700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc80c9b40 0x7ffbc80c8500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3478340506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc815d930 msgr2=0x7ffbc8153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.297+0000 7ffbc6ffd700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc815d930 0x7ffbc8153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7ffbc6ffd700 1 -- 172.21.15.3:0/3478340506 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc0007790 con 0x7ffbc81579a0 2023-11-04T05:28:48.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f133aa6d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13340c3530 0x7f13340d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f133aa6d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1334162160 0x7f13340ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f133aa6d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1334165a60 con 0x7f13340af620 2023-11-04T05:28:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f133aa6d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1334166300 con 0x7f13340c3530 2023-11-04T05:28:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f133aa6d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340b1730 con 0x7f1334162160 2023-11-04T05:28:48.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7ffbc6ffd700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc81579a0 0x7ffbc80c74e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffbc001d000 tx=0x7ffbc0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0015070 con 0x7ffbc81579a0 2023-11-04T05:28:48.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00193c0 con 0x7ffbc81579a0 2023-11-04T05:28:48.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0003a20 con 0x7ffbc81579a0 2023-11-04T05:28:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f1333fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f13340af620 0x7f13340c3910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57472/0 (socket says 172.21.15.3:57472) 2023-11-04T05:28:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f1333fff700 1 -- 172.21.15.3:0/3571243234 learned_addr learned my addr 172.21.15.3:0/3571243234 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f1338809700 1 --2- 172.21.15.3:0/3571243234 >> v2:172.21.15.3:3300/0 conn(0x7f13340c3530 0x7f13340d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f133900a700 1 --2- 172.21.15.3:0/3571243234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1334162160 0x7f13340ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.298+0000 7f09ae19e700 1 Processor -- start 2023-11-04T05:28:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f13337fe700 1 -- 172.21.15.3:0/3571243234 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712375778 0 0) 0x7f1334165a60 con 0x7f13340af620 2023-11-04T05:28:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f13337fe700 1 -- 172.21.15.3:0/3571243234 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13240035b0 con 0x7f13340af620 2023-11-04T05:28:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f1338809700 1 -- 172.21.15.3:0/3571243234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1334162160 msgr2=0x7f13340ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f1338809700 1 --2- 172.21.15.3:0/3571243234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1334162160 0x7f13340ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f1338809700 1 -- 172.21.15.3:0/3571243234 >> v1:172.21.15.3:6789/0 conn(0x7f13340af620 legacy=0x7f13340c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f1338809700 1 -- 172.21.15.3:0/3571243234 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1334165cb0 con 0x7f13340c3530 2023-11-04T05:28:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f1338809700 1 --2- 172.21.15.3:0/3571243234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340d1080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1320009820 tx=0x7f1320009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5706f101b5571072 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f13337fe700 1 -- 172.21.15.3:0/3571243234 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320005e30 con 0x7f13340c3530 2023-11-04T05:28:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f09ae19e700 1 -- start start 2023-11-04T05:28:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ae19e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09a80cf080 0x7f09a806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ae19e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a815a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ae19e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80cb820 con 0x7f09a815ac50 2023-11-04T05:28:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ae19e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a815bd50 con 0x7f09a80cf080 2023-11-04T05:28:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ae19e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a806ece0 con 0x7f09a80cf460 2023-11-04T05:28:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.296+0000 7fb234f47700 1 -- start start 2023-11-04T05:28:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09a80cf080 0x7f09a806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09a80cf080 0x7f09a806e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51218/0 (socket says 172.21.15.3:51218) 2023-11-04T05:28:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 -- 172.21.15.3:0/1263536790 learned_addr learned my addr 172.21.15.3:0/1263536790 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f13337fe700 1 -- 172.21.15.3:0/3571243234 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320004030 con 0x7f13340c3530 2023-11-04T05:28:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.299+0000 7f13337fe700 1 -- 172.21.15.3:0/3571243234 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200043c0 con 0x7f13340c3530 2023-11-04T05:28:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f133aa6d700 1 -- 172.21.15.3:0/3571243234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 msgr2=0x7f13340d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3571243234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340d1080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1320009820 tx=0x7f1320009b30 comp rx=0 tx=0).stop 2023-11-04T05:28:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ac99b700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 -- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 msgr2=0x7f09a815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 -- 172.21.15.3:0/1263536790 >> v1:172.21.15.188:6789/0 conn(0x7f09a815ac50 legacy=0x7f09a80cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ad19c700 1 -- 172.21.15.3:0/1263536790 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a815b1d0 con 0x7f09a80cf080 2023-11-04T05:28:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.300+0000 7f09ac99b700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7f09ad19c700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a806e7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009920 tx=0x7f09a4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3079500019bce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc80a8720 con 0x7ffbc81579a0 2023-11-04T05:28:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7f133aa6d700 1 -- 172.21.15.3:0/3571243234 shutdown_connections 2023-11-04T05:28:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3571243234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1334162160 0x7f13340ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3571243234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340d1080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7f133aa6d700 1 -- 172.21.15.3:0/3571243234 >> 172.21.15.3:0/3571243234 conn(0x7f13340d0560 msgr2=0x7f13340d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb234f47700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb23006c240 0x7fb2300721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb234f47700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2300cc0c0 0x7fb2300ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb234f47700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300cc4f0 con 0x7fb230165630 2023-11-04T05:28:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb234f47700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300ce210 con 0x7fb2300cc0c0 2023-11-04T05:28:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb234f47700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300cdc20 con 0x7fb23006c240 2023-11-04T05:28:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7f133aa6d700 1 -- 172.21.15.3:0/3571243234 shutdown_connections 2023-11-04T05:28:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc80c6780 con 0x7ffbc81579a0 2023-11-04T05:28:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7f133aa6d700 1 -- 172.21.15.3:0/3571243234 wait complete. 2023-11-04T05:28:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7f099ffff700 1 -- 172.21.15.3:0/1263536790 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400ba20 con 0x7f09a80cf080 2023-11-04T05:28:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7f099ffff700 1 -- 172.21.15.3:0/1263536790 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a400bb80 con 0x7f09a80cf080 2023-11-04T05:28:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7f099ffff700 1 -- 172.21.15.3:0/1263536790 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4009e70 con 0x7f09a80cf080 2023-11-04T05:28:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2300cc0c0 0x7fb2300ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2300cc0c0 0x7fb2300ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52138/0 (socket says 172.21.15.3:52138) 2023-11-04T05:28:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 -- 172.21.15.3:0/2040009476 learned_addr learned my addr 172.21.15.3:0/2040009476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 -- 172.21.15.3:0/2040009476 >> v2:172.21.15.188:3300/0 conn(0x7fb23006c240 msgr2=0x7fb2300721e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 --2- 172.21.15.3:0/2040009476 >> v2:172.21.15.188:3300/0 conn(0x7fb23006c240 0x7fb2300721e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 -- 172.21.15.3:0/2040009476 >> v1:172.21.15.188:6789/0 conn(0x7fb230165630 legacy=0x7fb23006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fb22ed9d700 1 -- 172.21.15.3:0/2040009476 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230066a80 con 0x7fb2300cc0c0 2023-11-04T05:28:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7fda0c688700 1 Processor -- start 2023-11-04T05:28:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.301+0000 7fda0c688700 1 -- start start 2023-11-04T05:28:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fda0c688700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda0416a9c0 0x7fda040cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fda0c688700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda040cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fda0c688700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040cfdf0 con 0x7fda0416e5c0 2023-11-04T05:28:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fda0c688700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040cab40 con 0x7fda0416a9c0 2023-11-04T05:28:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.302+0000 7fda0c688700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040ccda0 con 0x7fda040cf450 2023-11-04T05:28:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda0416a9c0 0x7fda040cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda0416a9c0 0x7fda040cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52140/0 (socket says 172.21.15.3:52140) 2023-11-04T05:28:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 -- 172.21.15.3:0/869846720 learned_addr learned my addr 172.21.15.3:0/869846720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fb22ed9d700 1 --2- 172.21.15.3:0/2040009476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300ce630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb2300cbe60 tx=0x7fb22800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844903f59a2418b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc80c6910 con 0x7ffbc81579a0 2023-11-04T05:28:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7f133aa6d700 1 Processor -- start 2023-11-04T05:28:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7f09ae19e700 1 -- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 msgr2=0x7f09a806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7f09ae19e700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a806e7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009920 tx=0x7f09a4009c30 comp rx=0 tx=0).stop 2023-11-04T05:28:48.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fb22d59a700 1 -- 172.21.15.3:0/2040009476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800e070 con 0x7fb2300cc0c0 2023-11-04T05:28:48.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fb22d59a700 1 -- 172.21.15.3:0/2040009476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228003a20 con 0x7fb2300cc0c0 2023-11-04T05:28:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fb22d59a700 1 -- 172.21.15.3:0/2040009476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228003dd0 con 0x7fb2300cc0c0 2023-11-04T05:28:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffbc0003b80 con 0x7ffbc81579a0 2023-11-04T05:28:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda09c23700 1 --2- 172.21.15.3:0/869846720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda040cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 -- 172.21.15.3:0/869846720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 msgr2=0x7fda040cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 --2- 172.21.15.3:0/869846720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda040cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 -- 172.21.15.3:0/869846720 >> v1:172.21.15.188:6789/0 conn(0x7fda0416e5c0 legacy=0x7fda0416ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.303+0000 7fda0a424700 1 -- 172.21.15.3:0/869846720 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040c9ec0 con 0x7fda0416a9c0 2023-11-04T05:28:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fda0a424700 1 --2- 172.21.15.3:0/869846720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416a9c0 0x7fda040cc860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4008370 tx=0x7fd9f4008680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aabfe48591309a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fda09422700 1 -- 172.21.15.3:0/869846720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4008cd0 con 0x7fda0416a9c0 2023-11-04T05:28:48.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fda09422700 1 -- 172.21.15.3:0/869846720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4008e30 con 0x7fda0416a9c0 2023-11-04T05:28:48.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7fda09422700 1 -- 172.21.15.3:0/869846720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4004250 con 0x7fda0416a9c0 2023-11-04T05:28:48.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f2294a37700 1 Processor -- start 2023-11-04T05:28:48.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f2294a37700 1 -- start start 2023-11-04T05:28:48.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7ffbaffff700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffbb406afd0 0x7ffbb406d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffbc008a7b0 con 0x7ffbc81579a0 2023-11-04T05:28:48.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7ffbc77fe700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffbb406afd0 0x7ffbb406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7fb234f47700 1 -- 172.21.15.3:0/2040009476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 msgr2=0x7fb2300ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7fb234f47700 1 --2- 172.21.15.3:0/2040009476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300ce630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb2300cbe60 tx=0x7fb22800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:48.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fb234f47700 1 -- 172.21.15.3:0/2040009476 shutdown_connections 2023-11-04T05:28:48.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fb234f47700 1 --2- 172.21.15.3:0/2040009476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300ce630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fb234f47700 1 --2- 172.21.15.3:0/2040009476 >> v2:172.21.15.188:3300/0 conn(0x7fb23006c240 0x7fb2300721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fb234f47700 1 -- 172.21.15.3:0/2040009476 >> 172.21.15.3:0/2040009476 conn(0x7fb2300716c0 msgr2=0x7fb230071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fb234f47700 1 -- 172.21.15.3:0/2040009476 shutdown_connections 2023-11-04T05:28:48.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.304+0000 7f133aa6d700 1 -- start start 2023-11-04T05:28:48.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f133aa6d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13340af620 0x7f13340ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f133aa6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f133aa6d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334162160 0x7f13340b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f133aa6d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340cd030 con 0x7f13340c3530 2023-11-04T05:28:48.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f133aa6d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340b6070 con 0x7f1334162160 2023-11-04T05:28:48.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7f133aa6d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340ca900 con 0x7f13340af620 2023-11-04T05:28:48.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f1333fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334162160 0x7f13340b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f1333fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52146/0 (socket says 172.21.15.3:52146) 2023-11-04T05:28:48.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f1333fff700 1 -- 172.21.15.3:0/3046334116 learned_addr learned my addr 172.21.15.3:0/3046334116 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f1338809700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13340af620 0x7f13340ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7fda0c688700 1 -- 172.21.15.3:0/869846720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416a9c0 msgr2=0x7fda040cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.305+0000 7fda0c688700 1 --2- 172.21.15.3:0/869846720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416a9c0 0x7fda040cc860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4008370 tx=0x7fd9f4008680 comp rx=0 tx=0).stop 2023-11-04T05:28:48.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fda0c688700 1 -- 172.21.15.3:0/869846720 shutdown_connections 2023-11-04T05:28:48.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fda0c688700 1 --2- 172.21.15.3:0/869846720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416a9c0 0x7fda040cc860 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4008370 tx=0x7fd9f4008680 comp rx=0 tx=0).stop 2023-11-04T05:28:48.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fda0c688700 1 --2- 172.21.15.3:0/869846720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda040cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fda0c688700 1 -- 172.21.15.3:0/869846720 >> 172.21.15.3:0/869846720 conn(0x7fda0406b150 msgr2=0x7fda0406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fda0c688700 1 -- 172.21.15.3:0/869846720 shutdown_connections 2023-11-04T05:28:48.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 -- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13340af620 msgr2=0x7f13340ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13340af620 0x7f13340ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 -- 172.21.15.3:0/3046334116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 msgr2=0x7f13340b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 -- 172.21.15.3:0/3046334116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009430 con 0x7f1334162160 2023-11-04T05:28:48.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7ff5081aa700 1 Processor -- start 2023-11-04T05:28:48.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fb234f47700 1 -- 172.21.15.3:0/2040009476 wait complete. 2023-11-04T05:28:48.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f1333fff700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:48.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f09ae19e700 1 -- 172.21.15.3:0/1263536790 shutdown_connections 2023-11-04T05:28:48.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f09ae19e700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a806e7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f09ae19e700 1 --2- 172.21.15.3:0/1263536790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f09ae19e700 1 -- 172.21.15.3:0/1263536790 >> 172.21.15.3:0/1263536790 conn(0x7f09a806bc30 msgr2=0x7f09a806c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7f133900a700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334162160 0x7f13340b5b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1334072100 tx=0x7f132c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7ff5081aa700 1 -- start start 2023-11-04T05:28:48.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f2294a37700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f229006b560 0x7f2290157910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f2294a37700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f229015d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f2294a37700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f229015de70 0x7f229015a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f2294a37700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229015e2e0 con 0x7f229015de70 2023-11-04T05:28:48.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f2294a37700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22900729a0 con 0x7f229006b560 2023-11-04T05:28:48.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f2294a37700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290071900 con 0x7f22900cc760 2023-11-04T05:28:48.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7ffbc77fe700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffbb406afd0 0x7ffbb406d490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8001590 tx=0x7ffbb8007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.306+0000 7fda0c688700 1 -- 172.21.15.3:0/869846720 wait complete. 2023-11-04T05:28:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f229015de70 0x7f229015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228e59c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f229006b560 0x7f2290157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f229015de70 0x7f229015a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52154/0 (socket says 172.21.15.3:52154) 2023-11-04T05:28:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228ed9d700 1 -- 172.21.15.3:0/1550428084 learned_addr learned my addr 172.21.15.3:0/1550428084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228dd9b700 1 --2- 172.21.15.3:0/1550428084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f229015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228ed9d700 1 -- 172.21.15.3:0/1550428084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 msgr2=0x7f229015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.307+0000 7f228ed9d700 1 --2- 172.21.15.3:0/1550428084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f229015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228ed9d700 1 -- 172.21.15.3:0/1550428084 >> v2:172.21.15.188:3300/0 conn(0x7f229006b560 msgr2=0x7f2290157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228ed9d700 1 --2- 172.21.15.3:0/1550428084 >> v2:172.21.15.188:3300/0 conn(0x7f229006b560 0x7f2290157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228ed9d700 1 -- 172.21.15.3:0/1550428084 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290071be0 con 0x7f229015de70 2023-11-04T05:28:48.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7fb234f47700 1 Processor -- start 2023-11-04T05:28:48.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff5081aa700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5000cbf00 0x7ff5000cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228ed9d700 1 --2- 172.21.15.3:0/1550428084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229015de70 0x7f229015a340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22900cd000 tx=0x7f228800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba189d4263d06b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7fb234f47700 1 -- start start 2023-11-04T05:28:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff5081aa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff5000d00b0 0x7ff500070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff5081aa700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5000cf380 con 0x7ff500071510 2023-11-04T05:28:48.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff5081aa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5000cc820 con 0x7ff5000d00b0 2023-11-04T05:28:48.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff5081aa700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff500071a50 con 0x7ff5000cbf00 2023-11-04T05:28:48.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7fda0c688700 1 Processor -- start 2023-11-04T05:28:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff506747700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff500071510 0x7ff5000ca060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:36048/0 (socket says 172.21.15.3:36048) 2023-11-04T05:28:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff506747700 1 -- 172.21.15.3:0/3357117871 learned_addr learned my addr 172.21.15.3:0/3357117871 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff505745700 1 --2- 172.21.15.3:0/3357117871 >> v2:172.21.15.3:3300/0 conn(0x7ff5000d00b0 0x7ff500070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228d59a700 1 -- 172.21.15.3:0/1550428084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800e070 con 0x7f229015de70 2023-11-04T05:28:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228d59a700 1 -- 172.21.15.3:0/1550428084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228800b910 con 0x7f229015de70 2023-11-04T05:28:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7f228d59a700 1 -- 172.21.15.3:0/1550428084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800bcc0 con 0x7f229015de70 2023-11-04T05:28:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7f2294a37700 1 -- 172.21.15.3:0/1550428084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229015de70 msgr2=0x7f229015a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7f2294a37700 1 --2- 172.21.15.3:0/1550428084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229015de70 0x7f229015a340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22900cd000 tx=0x7f228800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff505f46700 1 --2- 172.21.15.3:0/3357117871 >> v2:172.21.15.188:3300/0 conn(0x7ff5000cbf00 0x7ff5000cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff505745700 1 -- 172.21.15.3:0/3357117871 >> v2:172.21.15.188:3300/0 conn(0x7ff5000cbf00 msgr2=0x7ff5000cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff505745700 1 --2- 172.21.15.3:0/3357117871 >> v2:172.21.15.188:3300/0 conn(0x7ff5000cbf00 0x7ff5000cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff505745700 1 -- 172.21.15.3:0/3357117871 >> v1:172.21.15.188:6789/0 conn(0x7ff500071510 legacy=0x7ff5000ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7ff505745700 1 -- 172.21.15.3:0/3357117871 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5000cf5d0 con 0x7ff5000d00b0 2023-11-04T05:28:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7ff505745700 1 --2- 172.21.15.3:0/3357117871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000d00b0 0x7ff500070fd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5000d0560 tx=0x7ff4f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df02d0f667c1ffff server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7ff504f44700 1 -- 172.21.15.3:0/3357117871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0011830 con 0x7ff5000d00b0 2023-11-04T05:28:48.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7f09ae19e700 1 -- 172.21.15.3:0/1263536790 shutdown_connections 2023-11-04T05:28:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb234f47700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb23006c240 0x7fb23014e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb234f47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb234f47700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 0x7fb2300b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb234f47700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23014e540 con 0x7fb2300cc0c0 2023-11-04T05:28:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb234f47700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c5340 con 0x7fb230165630 2023-11-04T05:28:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb234f47700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c5620 con 0x7fb23006c240 2023-11-04T05:28:48.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7f09ae19e700 1 -- 172.21.15.3:0/1263536790 wait complete. 2023-11-04T05:28:48.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7ff504f44700 1 -- 172.21.15.3:0/3357117871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0011990 con 0x7ff5000d00b0 2023-11-04T05:28:48.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7ff504f44700 1 -- 172.21.15.3:0/3357117871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0011d20 con 0x7ff5000d00b0 2023-11-04T05:28:48.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.308+0000 7fda0c688700 1 -- start start 2023-11-04T05:28:48.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fda0c688700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda042044a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fda0c688700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 0x7fda040b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fda0c688700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda040b4550 0x7fda040b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fda0c688700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04204010 con 0x7fda0416e5c0 2023-11-04T05:28:48.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fda0c688700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040b5000 con 0x7fda040b4550 2023-11-04T05:28:48.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fda0c688700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040b4a40 con 0x7fda040cf450 2023-11-04T05:28:48.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 -- 172.21.15.3:0/1550428084 shutdown_connections 2023-11-04T05:28:48.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 --2- 172.21.15.3:0/1550428084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229015de70 0x7f229015a340 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 --2- 172.21.15.3:0/1550428084 >> v2:172.21.15.188:3300/0 conn(0x7f229006b560 0x7f2290157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 --2- 172.21.15.3:0/1550428084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f229015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 -- 172.21.15.3:0/1550428084 >> 172.21.15.3:0/1550428084 conn(0x7f229006b150 msgr2=0x7f229006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 -- 172.21.15.3:0/1550428084 shutdown_connections 2023-11-04T05:28:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f2294a37700 1 -- 172.21.15.3:0/1550428084 wait complete. 2023-11-04T05:28:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7f09ae19e700 1 Processor -- start 2023-11-04T05:28:48.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7f09ae19e700 1 -- start start 2023-11-04T05:28:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ffbc00555e0 con 0x7ffbc81579a0 2023-11-04T05:28:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb22ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 0x7fb2300b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb22ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 0x7fb2300b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51258/0 (socket says 172.21.15.3:51258) 2023-11-04T05:28:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.309+0000 7fb22ed9d700 1 -- 172.21.15.3:0/4043046552 learned_addr learned my addr 172.21.15.3:0/4043046552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fb22dd9b700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fb22ed9d700 1 -- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb23006c240 msgr2=0x7fb23014e000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-04T05:28:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fb22ed9d700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb23006c240 0x7fb23014e000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fb22ed9d700 1 -- 172.21.15.3:0/4043046552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 msgr2=0x7fb2300c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fb22ed9d700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fb22ed9d700 1 -- 172.21.15.3:0/4043046552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009b30 con 0x7fb230165630 2023-11-04T05:28:48.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fb22ed9d700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 0x7fb2300b2690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb230067890 tx=0x7fb228004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7ff5081aa700 1 -- 172.21.15.3:0/3357117871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000d00b0 msgr2=0x7ff500070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3357117871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000d00b0 0x7ff500070fd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5000d0560 tx=0x7ff4f0009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:48.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7ff5081aa700 1 -- 172.21.15.3:0/3357117871 shutdown_connections 2023-11-04T05:28:48.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3357117871 >> v2:172.21.15.188:3300/0 conn(0x7ff5000cbf00 0x7ff5000cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3357117871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5000d00b0 0x7ff500070fd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7ff5081aa700 1 -- 172.21.15.3:0/3357117871 >> 172.21.15.3:0/3357117871 conn(0x7ff50006b150 msgr2=0x7ff50006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7ff5081aa700 1 -- 172.21.15.3:0/3357117871 shutdown_connections 2023-11-04T05:28:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7ff5081aa700 1 -- 172.21.15.3:0/3357117871 wait complete. 2023-11-04T05:28:48.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00cc20 con 0x7f1334162160 2023-11-04T05:28:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c00f040 con 0x7f1334162160 2023-11-04T05:28:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c009e60 con 0x7f1334162160 2023-11-04T05:28:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13340cc690 con 0x7f1334162160 2023-11-04T05:28:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13340ccc50 con 0x7f1334162160 2023-11-04T05:28:48.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fda09c23700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 0x7fda040b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fda09c23700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 0x7fda040b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52172/0 (socket says 172.21.15.3:52172) 2023-11-04T05:28:48.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fda09c23700 1 -- 172.21.15.3:0/3637649792 learned_addr learned my addr 172.21.15.3:0/3637649792 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fda0ac25700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda040b4550 0x7fda040b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.310+0000 7fda0a424700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda042044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13340cd170 con 0x7f1334162160 2023-11-04T05:28:48.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f2294a37700 1 Processor -- start 2023-11-04T05:28:48.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228004440 con 0x7fb230165630 2023-11-04T05:28:48.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2280045a0 con 0x7fb230165630 2023-11-04T05:28:48.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228004950 con 0x7fb230165630 2023-11-04T05:28:48.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300c5900 con 0x7fb230165630 2023-11-04T05:28:48.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda0ac25700 1 -- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 msgr2=0x7fda042044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda0ac25700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda042044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda0ac25700 1 -- 172.21.15.3:0/3637649792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 msgr2=0x7fda040b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda0ac25700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 0x7fda040b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda0ac25700 1 -- 172.21.15.3:0/3637649792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4008050 con 0x7fda040b4550 2023-11-04T05:28:48.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda09c23700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 0x7fda040b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:48.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.311+0000 7fda0ac25700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda040b4550 0x7fda040b2100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c780 tx=0x7fd9fc00ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003e10 con 0x7fda040b4550 2023-11-04T05:28:48.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc0050d0 con 0x7fda040b4550 2023-11-04T05:28:48.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc005480 con 0x7fda040b4550 2023-11-04T05:28:48.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f489b54c700 1 Processor -- start 2023-11-04T05:28:48.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f09ae19e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a81ec640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f09ae19e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a81ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f09ae19e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a815ac50 0x7f09a814d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f09ae19e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81ecb80 con 0x7f09a815ac50 2023-11-04T05:28:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f09ae19e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80c72a0 con 0x7f09a80cf080 2023-11-04T05:28:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f09ae19e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81efa50 con 0x7f09a80cf460 2023-11-04T05:28:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad99d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a815ac50 0x7f09a814d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad99d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a815ac50 0x7f09a814d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52182/0 (socket says 172.21.15.3:52182) 2023-11-04T05:28:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a81ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad99d700 1 -- 172.21.15.3:0/3725354835 learned_addr learned my addr 172.21.15.3:0/3725354835 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 -- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 msgr2=0x7f09a81ef0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-04T05:28:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a81ef0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 -- 172.21.15.3:0/3725354835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a815ac50 msgr2=0x7f09a814d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a815ac50 0x7f09a814d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 -- 172.21.15.3:0/3725354835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a4009550 con 0x7f09a80cf080 2023-11-04T05:28:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda040b4d20 con 0x7fda040b4550 2023-11-04T05:28:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f09ad19c700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a81ec640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09a4016000 tx=0x7f09a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f132c014480 con 0x7f1334162160 2023-11-04T05:28:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f13317fa700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f131806af20 0x7f131806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f132c08a620 con 0x7f1334162160 2023-11-04T05:28:48.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7ff5081aa700 1 Processor -- start 2023-11-04T05:28:48.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7ff5081aa700 1 -- start start 2023-11-04T05:28:48.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff5081aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 0x7ff5000b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff5081aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5000cbf00 0x7ff5000c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff5081aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5000d00b0 0x7ff5000c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff5081aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000b3020 con 0x7ff500071510 2023-11-04T05:28:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff5081aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50015a0c0 con 0x7ff5000d00b0 2023-11-04T05:28:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff5081aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50015a3a0 con 0x7ff5000cbf00 2023-11-04T05:28:48.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f1338809700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f131806af20 0x7f131806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505745700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5000cbf00 0x7ff5000c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505f46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 0x7ff5000b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505f46700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 0x7ff5000b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52192/0 (socket says 172.21.15.3:52192) 2023-11-04T05:28:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505f46700 1 -- 172.21.15.3:0/3261633434 learned_addr learned my addr 172.21.15.3:0/3261633434 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300c5ec0 con 0x7fb230165630 2023-11-04T05:28:48.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff506747700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5000d00b0 0x7ff5000c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505f46700 1 -- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5000cbf00 msgr2=0x7ff5000c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505f46700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5000cbf00 0x7ff5000c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7ff505f46700 1 -- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5000d00b0 msgr2=0x7ff5000c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7ff505f46700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5000d00b0 0x7ff5000c3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7ff505f46700 1 -- 172.21.15.3:0/3261633434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00094a0 con 0x7ff500071510 2023-11-04T05:28:48.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7ff505f46700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 0x7ff5000b27f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e9b0 tx=0x7ff4fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f489b54c700 1 -- start start 2023-11-04T05:28:48.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f1338809700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f131806af20 0x7f131806d3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1320009b70 tx=0x7f1320008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda042049e0 con 0x7fda040b4550 2023-11-04T05:28:48.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda04204180 con 0x7fda040b4550 2023-11-04T05:28:48.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.312+0000 7f2294a37700 1 -- start start 2023-11-04T05:28:48.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f2294a37700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 0x7f229014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f2294a37700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f22901487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f2294a37700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f229015de70 0x7f2290148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f2294a37700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290159970 con 0x7f229006b560 2023-11-04T05:28:48.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f2294a37700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c6780 con 0x7f229015de70 2023-11-04T05:28:48.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.313+0000 7f2294a37700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c5890 con 0x7f22900cc760 2023-11-04T05:28:48.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 0x7f229014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 0x7f229014a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52184/0 (socket says 172.21.15.3:52184) 2023-11-04T05:28:48.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 -- 172.21.15.3:0/3124774376 learned_addr learned my addr 172.21.15.3:0/3124774376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 -- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 msgr2=0x7f22901487e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-04T05:28:48.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f22901487e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 -- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f229015de70 msgr2=0x7f2290148d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-04T05:28:48.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f229015de70 0x7f2290148d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 -- 172.21.15.3:0/3124774376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288009b30 con 0x7f229006b560 2023-11-04T05:28:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.314+0000 7f228e59c700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 0x7f229014a360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f228000e960 tx=0x7f228000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000c850 con 0x7f229006b560 2023-11-04T05:28:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228000c9b0 con 0x7f229006b560 2023-11-04T05:28:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22901593b0 con 0x7f229006b560 2023-11-04T05:28:48.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000cd20 con 0x7f229006b560 2023-11-04T05:28:48.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.315+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290159080 con 0x7f229006b560 2023-11-04T05:28:48.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230164440 con 0x7fb230165630 2023-11-04T05:28:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc009710 con 0x7ff500071510 2023-11-04T05:28:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f489b54c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f489415de90 0x7f48940ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f489b54c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4894072df0 0x7f48940cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f489b54c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48940cdd70 con 0x7f48940cf730 2023-11-04T05:28:48.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f489b54c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48940cfbe0 con 0x7f4894072df0 2023-11-04T05:28:48.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f489b54c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48940cd160 con 0x7f489415de90 2023-11-04T05:28:48.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00c850 con 0x7ff500071510 2023-11-04T05:28:48.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00cbc0 con 0x7ff500071510 2023-11-04T05:28:48.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50015a680 con 0x7ff500071510 2023-11-04T05:28:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f48992e8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f489415de90 0x7f48940ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f48992e8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f489415de90 0x7f48940ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51284/0 (socket says 172.21.15.3:51284) 2023-11-04T05:28:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f48992e8700 1 -- 172.21.15.3:0/403578227 learned_addr learned my addr 172.21.15.3:0/403578227 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7f4898ae7700 1 --2- 172.21.15.3:0/403578227 >> v2:172.21.15.3:3300/0 conn(0x7f4894072df0 0x7f48940cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f48992e8700 1 -- 172.21.15.3:0/403578227 >> v2:172.21.15.3:3300/0 conn(0x7f4894072df0 msgr2=0x7f48940cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f48992e8700 1 --2- 172.21.15.3:0/403578227 >> v2:172.21.15.3:3300/0 conn(0x7f4894072df0 0x7f48940cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f48992e8700 1 -- 172.21.15.3:0/403578227 >> v1:172.21.15.3:6789/0 conn(0x7f48940cf730 legacy=0x7f489415e630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f48992e8700 1 -- 172.21.15.3:0/403578227 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940cd440 con 0x7f489415de90 2023-11-04T05:28:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f48992e8700 1 --2- 172.21.15.3:0/403578227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f489415de90 0x7f48940ae740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f489406f0a0 tx=0x7f4890009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e29fb3033b0654 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4004510 con 0x7f09a80cf080 2023-11-04T05:28:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a81efd30 con 0x7f09a80cf080 2023-11-04T05:28:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290074df0 con 0x7f229006b560 2023-11-04T05:28:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.317+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a814d0d0 con 0x7f09a80cf080 2023-11-04T05:28:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2280013470 con 0x7f229006b560 2023-11-04T05:28:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f132c055550 con 0x7f1334162160 2023-11-04T05:28:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f488bfff700 1 -- 172.21.15.3:0/403578227 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890004230 con 0x7f489415de90 2023-11-04T05:28:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a40036a0 con 0x7f09a80cf080 2023-11-04T05:28:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4003a10 con 0x7f09a80cf080 2023-11-04T05:28:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f488bfff700 1 -- 172.21.15.3:0/403578227 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4890004390 con 0x7f489415de90 2023-11-04T05:28:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f488bfff700 1 -- 172.21.15.3:0/403578227 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900036a0 con 0x7f489415de90 2023-11-04T05:28:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f227effd700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af50 0x7f227006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.316+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5000b2d30 con 0x7ff500071510 2023-11-04T05:28:48.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.318+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2280089790 con 0x7f229006b560 2023-11-04T05:28:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.319+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb228026450 con 0x7fb230165630 2023-11-04T05:28:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.319+0000 7f489b54c700 1 -- 172.21.15.3:0/403578227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f489415de90 msgr2=0x7f48940ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.319+0000 7f489b54c700 1 --2- 172.21.15.3:0/403578227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f489415de90 0x7f48940ae740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f489406f0a0 tx=0x7f4890009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.319+0000 7fb21effd700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb21006af50 0x7fb21006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a8073010 con 0x7f09a80cf080 2023-11-04T05:28:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb22808ba90 con 0x7fb230165630 2023-11-04T05:28:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09a4003b70 con 0x7f09a80cf080 2023-11-04T05:28:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f099dffb700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f098c06af00 0x7f098c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f489b54c700 1 -- 172.21.15.3:0/403578227 shutdown_connections 2023-11-04T05:28:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f489b54c700 1 --2- 172.21.15.3:0/403578227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f489415de90 0x7f48940ae740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f489b54c700 1 --2- 172.21.15.3:0/403578227 >> v2:172.21.15.3:3300/0 conn(0x7f4894072df0 0x7f48940cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f489b54c700 1 -- 172.21.15.3:0/403578227 >> 172.21.15.3:0/403578227 conn(0x7f489406b4d0 msgr2=0x7f489406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f09ac99b700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f098c06af00 0x7f098c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f489b54c700 1 -- 172.21.15.3:0/403578227 shutdown_connections 2023-11-04T05:28:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.320+0000 7f489b54c700 1 -- 172.21.15.3:0/403578227 wait complete. 2023-11-04T05:28:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.321+0000 7f09ac99b700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f098c06af00 0x7f098c06d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09a81f3790 tx=0x7f0998008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.322+0000 7f489b54c700 1 Processor -- start 2023-11-04T05:28:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.322+0000 7f489b54c700 1 -- start start 2023-11-04T05:28:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f228dd9b700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af50 0x7f227006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f489b54c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4894072df0 0x7f4894152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f489b54c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 0x7f4894203fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f489b54c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f489415de90 0x7f489414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f489b54c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48941529a0 con 0x7f4894072df0 2023-11-04T05:28:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f489b54c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894151710 con 0x7f48940cf730 2023-11-04T05:28:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7f489b54c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4894151980 con 0x7f489415de90 2023-11-04T05:28:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.323+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9fc0055e0 con 0x7fda040b4550 2023-11-04T05:28:48.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 0x7f4894203fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 0x7f4894203fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51294/0 (socket says 172.21.15.3:51294) 2023-11-04T05:28:48.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 -- 172.21.15.3:0/3671345199 learned_addr learned my addr 172.21.15.3:0/3671345199 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:48.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f228dd9b700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af50 0x7f227006d410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2290159d90 tx=0x7f2284009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fb22805a350 con 0x7fb230165630 2023-11-04T05:28:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 -- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f489415de90 msgr2=0x7f489414f5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-04T05:28:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f489415de90 0x7f489414f5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 -- 172.21.15.3:0/3671345199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4894072df0 msgr2=0x7f4894152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4894072df0 0x7f4894152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 -- 172.21.15.3:0/3671345199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890009550 con 0x7f48940cf730 2023-11-04T05:28:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7fd9faffd700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd9e806afd0 0x7fd9e806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd9fc089370 con 0x7fda040b4550 2023-11-04T05:28:48.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7fda0a424700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd9e806afd0 0x7fd9e806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.324+0000 7f4898ae7700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 0x7f4894203fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48840094a0 tx=0x7f4884009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.325+0000 7fda0a424700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd9e806afd0 0x7fd9e806d490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda040ccf80 tx=0x7fd9f4014000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.326+0000 7fb22e59c700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb21006af50 0x7fb21006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.326+0000 7fb22e59c700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb21006af50 0x7fb21006d410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224006bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.328+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e80052f0 con 0x7ff500071510 2023-11-04T05:28:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.328+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400d370 con 0x7f48940cf730 2023-11-04T05:28:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.328+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4894151c60 con 0x7f48940cf730 2023-11-04T05:28:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.328+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f09a4003e20 con 0x7f09a80cf080 2023-11-04T05:28:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.328+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48942011a0 con 0x7f48940cf730 2023-11-04T05:28:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.330+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4fc00cd20 con 0x7ff500071510 2023-11-04T05:28:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.330+0000 7ff4f67fc700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff4e406af50 0x7ff4e406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.331+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff4fc08a070 con 0x7ff500071510 2023-11-04T05:28:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.331+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4894152180 con 0x7f48940cf730 2023-11-04T05:28:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.331+0000 7ff505745700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff4e406af50 0x7ff4e406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.331+0000 7ff505745700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff4e406af50 0x7ff4e406d410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00090b0 tx=0x7ff4f001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.333+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd9fc054110 con 0x7fda040b4550 2023-11-04T05:28:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.333+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400de50 con 0x7f48940cf730 2023-11-04T05:28:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.333+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884005b40 con 0x7f48940cf730 2023-11-04T05:28:48.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.333+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f09a40558e0 con 0x7f09a80cf080 2023-11-04T05:28:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.335+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4884014070 con 0x7f48940cf730 2023-11-04T05:28:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.335+0000 7f4889ffb700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f487806af00 0x7f487806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.336+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4884089800 con 0x7f48940cf730 2023-11-04T05:28:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.336+0000 7f48992e8700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f487806af00 0x7f487806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.337+0000 7f48992e8700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f487806af00 0x7f487806d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4890009c10 tx=0x7f4890009f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.338+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4884054650 con 0x7f48940cf730 2023-11-04T05:28:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.339+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f22800546c0 con 0x7f229006b560 2023-11-04T05:28:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.345+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff4fc054f10 con 0x7ff500071510 2023-11-04T05:28:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.715+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffbc80c6910 con 0x7ffbc81579a0 2023-11-04T05:28:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.716+0000 7ffbaffff700 1 -- 172.21.15.3:0/3478340506 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbc0058c00 con 0x7ffbc81579a0 2023-11-04T05:28:48.721 INFO:teuthology.orchestra.run.smithi003.stdout:64424509450 2023-11-04T05:28:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffbb406afd0 msgr2=0x7ffbb406d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffbb406afd0 0x7ffbb406d490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8001590 tx=0x7ffbb8007a20 comp rx=0 tx=0).stop 2023-11-04T05:28:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc81579a0 msgr2=0x7ffbc80c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc81579a0 0x7ffbc80c74e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffbc001d000 tx=0x7ffbc0004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 shutdown_connections 2023-11-04T05:28:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffbb406afd0 0x7ffbb406d490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbc815d930 0x7ffbc8153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc80c9b40 0x7ffbc80c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 --2- 172.21.15.3:0/3478340506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc81579a0 0x7ffbc80c74e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.720+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 >> 172.21.15.3:0/3478340506 conn(0x7ffbc806b150 msgr2=0x7ffbc80cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.721+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 shutdown_connections 2023-11-04T05:28:48.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.721+0000 7ffbce039700 1 -- 172.21.15.3:0/3478340506 wait complete. 2023-11-04T05:28:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.730+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb230164160 con 0x7fb230165630 2023-11-04T05:28:48.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.731+0000 7fb21effd700 1 -- 172.21.15.3:0/4043046552 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb228059ee0 con 0x7fb230165630 2023-11-04T05:28:48.734 INFO:teuthology.orchestra.run.smithi003.stdout:60129542154 2023-11-04T05:28:48.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.733+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f13340cd170 con 0x7f1334162160 2023-11-04T05:28:48.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.734+0000 7f13317fa700 1 -- 172.21.15.3:0/3046334116 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f13340cd170 con 0x7f1334162160 2023-11-04T05:28:48.736 INFO:teuthology.orchestra.run.smithi003.stdout:64424509451 2023-11-04T05:28:48.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb21006af50 msgr2=0x7fb21006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb21006af50 0x7fb21006d410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224006bc0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 msgr2=0x7fb2300b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 0x7fb2300b2690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb230067890 tx=0x7fb228004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 shutdown_connections 2023-11-04T05:28:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb21006af50 0x7fb21006d410 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224006bc0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb230165630 0x7fb2300b2690 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb23006c240 0x7fb23014e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 --2- 172.21.15.3:0/4043046552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2300cc0c0 0x7fb2300c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 >> 172.21.15.3:0/4043046552 conn(0x7fb2300716c0 msgr2=0x7fb2301620d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 shutdown_connections 2023-11-04T05:28:48.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.735+0000 7fb234f47700 1 -- 172.21.15.3:0/4043046552 wait complete. 2023-11-04T05:28:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.739+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f131806af20 msgr2=0x7f131806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.739+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f131806af20 0x7f131806d3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1320009b70 tx=0x7f1320008070 comp rx=0 tx=0).stop 2023-11-04T05:28:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.739+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334162160 msgr2=0x7f13340b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.739+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334162160 0x7f13340b5b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1334072100 tx=0x7f132c00ef90 comp rx=0 tx=0).stop 2023-11-04T05:28:48.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.5 2023-11-04T05:28:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.740+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 shutdown_connections 2023-11-04T05:28:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.740+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f131806af20 0x7f131806d3e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.740+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334162160 0x7f13340b5b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.740+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13340af620 0x7f13340ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.740+0000 7f133aa6d700 1 --2- 172.21.15.3:0/3046334116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13340c3530 0x7f13340b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.740+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 >> 172.21.15.3:0/3046334116 conn(0x7f13340d0560 msgr2=0x7f133406f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.741+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 shutdown_connections 2023-11-04T05:28:48.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.741+0000 7f133aa6d700 1 -- 172.21.15.3:0/3046334116 wait complete. 2023-11-04T05:28:48.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.748+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda04074df0 con 0x7fda040b4550 2023-11-04T05:28:48.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.748+0000 7fd9faffd700 1 -- 172.21.15.3:0/3637649792 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd9fc017020 con 0x7fda040b4550 2023-11-04T05:28:48.750 INFO:teuthology.orchestra.run.smithi003.stdout:60129542153 2023-11-04T05:28:48.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.749+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f09a804f730 con 0x7f09a80cf080 2023-11-04T05:28:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd9e806afd0 msgr2=0x7fd9e806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd9e806afd0 0x7fd9e806d490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda040ccf80 tx=0x7fd9f4014000 comp rx=0 tx=0).stop 2023-11-04T05:28:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda040b4550 msgr2=0x7fda040b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda040b4550 0x7fda040b2100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c780 tx=0x7fd9fc00ca90 comp rx=0 tx=0).stop 2023-11-04T05:28:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 shutdown_connections 2023-11-04T05:28:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd9e806afd0 0x7fd9e806d490 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda040b4550 0x7fda040b2100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda040cf450 0x7fda042044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 --2- 172.21.15.3:0/3637649792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda0416e5c0 0x7fda040b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.752+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 >> 172.21.15.3:0/3637649792 conn(0x7fda0406b150 msgr2=0x7fda0408a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.753+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 shutdown_connections 2023-11-04T05:28:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.753+0000 7fda0c688700 1 -- 172.21.15.3:0/3637649792 wait complete. 2023-11-04T05:28:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.753+0000 7f099dffb700 1 -- 172.21.15.3:0/3725354835 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09a4028090 con 0x7f09a80cf080 2023-11-04T05:28:48.758 INFO:teuthology.orchestra.run.smithi003.stdout:60129542154 2023-11-04T05:28:48.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.3 2023-11-04T05:28:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f098c06af00 msgr2=0x7f098c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f098c06af00 0x7f098c06d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09a81f3790 tx=0x7f0998008040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 msgr2=0x7f09a81ec640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a81ec640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09a4016000 tx=0x7f09a4004750 comp rx=0 tx=0).stop 2023-11-04T05:28:48.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 shutdown_connections 2023-11-04T05:28:48.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f098c06af00 0x7f098c06d3c0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09a81f3790 tx=0x7f0998008040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a80cf080 0x7f09a81ec640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a815ac50 0x7f09a814d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 --2- 172.21.15.3:0/3725354835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80cf460 0x7f09a81ef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.757+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 >> 172.21.15.3:0/3725354835 conn(0x7f09a806bc30 msgr2=0x7f09a8159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.760+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 shutdown_connections 2023-11-04T05:28:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.760+0000 7f09ae19e700 1 -- 172.21.15.3:0/3725354835 wait complete. 2023-11-04T05:28:48.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.6 2023-11-04T05:28:48.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.762+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4894152180 con 0x7f48940cf730 2023-11-04T05:28:48.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.763+0000 7f4889ffb700 1 -- 172.21.15.3:0/3671345199 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4894152180 con 0x7f48940cf730 2023-11-04T05:28:48.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.763+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22900cd780 con 0x7f229006b560 2023-11-04T05:28:48.767 INFO:teuthology.orchestra.run.smithi003.stdout:64424509450 2023-11-04T05:28:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.763+0000 7f227effd700 1 -- 172.21.15.3:0/3124774376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2280057ce0 con 0x7f229006b560 2023-11-04T05:28:48.768 INFO:teuthology.orchestra.run.smithi003.stdout:60129542154 2023-11-04T05:28:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af50 msgr2=0x7f227006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f2294a37700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af50 0x7f227006d410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2290159d90 tx=0x7f2284009210 comp rx=0 tx=0).stop 2023-11-04T05:28:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 msgr2=0x7f229014a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f2294a37700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 0x7f229014a360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f228000e960 tx=0x7f228000ed20 comp rx=0 tx=0).stop 2023-11-04T05:28:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f487806af00 msgr2=0x7f487806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f487806af00 0x7f487806d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4890009c10 tx=0x7f4890009f90 comp rx=0 tx=0).stop 2023-11-04T05:28:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 msgr2=0x7f4894203fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 0x7f4894203fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48840094a0 tx=0x7f4884009860 comp rx=0 tx=0).stop 2023-11-04T05:28:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 shutdown_connections 2023-11-04T05:28:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f487806af00 0x7f487806d3c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f489415de90 0x7f489414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4894072df0 0x7f4894152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 --2- 172.21.15.3:0/3671345199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cf730 0x7f4894203fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.766+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 >> 172.21.15.3:0/3671345199 conn(0x7f489406b4d0 msgr2=0x7f48940c5140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.767+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 shutdown_connections 2023-11-04T05:28:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.767+0000 7f489b54c700 1 -- 172.21.15.3:0/3671345199 wait complete. 2023-11-04T05:28:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 shutdown_connections 2023-11-04T05:28:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af50 0x7f227006d410 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f229015de70 0x7f2290148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f229006b560 0x7f229014a360 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 --2- 172.21.15.3:0/3124774376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22900cc760 0x7f22901487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 >> 172.21.15.3:0/3124774376 conn(0x7f229006b150 msgr2=0x7f229015b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 shutdown_connections 2023-11-04T05:28:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.768+0000 7f2294a37700 1 -- 172.21.15.3:0/3124774376 wait complete. 2023-11-04T05:28:48.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.1 2023-11-04T05:28:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.771+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff4e8005870 con 0x7ff500071510 2023-11-04T05:28:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.772+0000 7ff4f67fc700 1 -- 172.21.15.3:0/3261633434 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4fc017090 con 0x7ff500071510 2023-11-04T05:28:48.776 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2023-11-04T05:28:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff4e406af50 msgr2=0x7ff4e406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff4e406af50 0x7ff4e406d410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00090b0 tx=0x7ff4f001c040 comp rx=0 tx=0).stop 2023-11-04T05:28:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 msgr2=0x7ff5000b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 0x7ff5000b27f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e9b0 tx=0x7ff4fc00ecc0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 shutdown_connections 2023-11-04T05:28:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff4e406af50 0x7ff4e406d410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5000d00b0 0x7ff5000c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff500071510 0x7ff5000b27f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 --2- 172.21.15.3:0/3261633434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5000cbf00 0x7ff5000c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 >> 172.21.15.3:0/3261633434 conn(0x7ff50006b150 msgr2=0x7ff5000ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 shutdown_connections 2023-11-04T05:28:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:48.774+0000 7ff5081aa700 1 -- 172.21.15.3:0/3261633434 wait complete. 2023-11-04T05:28:48.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.2 2023-11-04T05:28:48.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.0 2023-11-04T05:28:48.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.7 2023-11-04T05:28:48.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2023-11-04T05:28:49.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-04T05:28:49.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-04T05:28:49.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-04T05:28:49.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-04T05:28:49.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-04T05:28:49.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-04T05:28:49.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-04T05:28:49.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-04T05:28:49.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.957+0000 7fd2df8fd700 1 Processor -- start 2023-11-04T05:28:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.957+0000 7fd2df8fd700 1 -- start start 2023-11-04T05:28:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.959+0000 7fd2df8fd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d815e040 0x7fd2d8166ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.959+0000 7fd2df8fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d8157da0 con 0x7fd2d8157910 2023-11-04T05:28:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.959+0000 7fd2df8fd700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80729a0 con 0x7fd2d806b560 2023-11-04T05:28:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.959+0000 7fd2df8fd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8071900 con 0x7fd2d815e040 2023-11-04T05:28:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.960+0000 7fd2dde9a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d815e040 0x7fd2d8166ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.960+0000 7fd2dce98700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd2d8157910 0x7fd2d815d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57566/0 (socket says 172.21.15.3:57566) 2023-11-04T05:28:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.960+0000 7fd2dce98700 1 -- 172.21.15.3:0/4227118164 learned_addr learned my addr 172.21.15.3:0/4227118164 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.961+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757423452 0 0) 0x7fd2d8157da0 con 0x7fd2d8157910 2023-11-04T05:28:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.961+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c40035b0 con 0x7fd2d8157910 2023-11-04T05:28:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.961+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916220818 0 0) 0x7fd2d80729a0 con 0x7fd2d806b560 2023-11-04T05:28:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.961+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d8157da0 con 0x7fd2d806b560 2023-11-04T05:28:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.962+0000 7fd2dde9a700 1 -- 172.21.15.3:0/4227118164 >> v1:172.21.15.188:6789/0 conn(0x7fd2d806b560 legacy=0x7fd2d80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.962+0000 7fd2dde9a700 1 -- 172.21.15.3:0/4227118164 >> v1:172.21.15.3:6789/0 conn(0x7fd2d8157910 legacy=0x7fd2d815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.962+0000 7fd2dde9a700 1 -- 172.21.15.3:0/4227118164 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8071be0 con 0x7fd2d815e040 2023-11-04T05:28:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.962+0000 7fd2dde9a700 1 --2- 172.21.15.3:0/4227118164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d815e040 0x7fd2d8166ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009dc0 tx=0x7fd2d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704662abf8d84521 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.963+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d000e070 con 0x7fd2d815e040 2023-11-04T05:28:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.963+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d000bb20 con 0x7fd2d815e040 2023-11-04T05:28:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.963+0000 7fd2cffff700 1 -- 172.21.15.3:0/4227118164 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0012430 con 0x7fd2d815e040 2023-11-04T05:28:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.964+0000 7fd2df8fd700 1 -- 172.21.15.3:0/4227118164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d815e040 msgr2=0x7fd2d8166ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.964+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/4227118164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d815e040 0x7fd2d8166ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009dc0 tx=0x7fd2d000b330 comp rx=0 tx=0).stop 2023-11-04T05:28:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.964+0000 7fd2df8fd700 1 -- 172.21.15.3:0/4227118164 shutdown_connections 2023-11-04T05:28:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.964+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/4227118164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d815e040 0x7fd2d8166ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.964+0000 7fd2df8fd700 1 -- 172.21.15.3:0/4227118164 >> 172.21.15.3:0/4227118164 conn(0x7fd2d806b150 msgr2=0x7fd2d806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:49.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.970+0000 7fd2df8fd700 1 -- 172.21.15.3:0/4227118164 shutdown_connections 2023-11-04T05:28:49.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.970+0000 7fd2df8fd700 1 -- 172.21.15.3:0/4227118164 wait complete. 2023-11-04T05:28:49.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.972+0000 7fd2df8fd700 1 Processor -- start 2023-11-04T05:28:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.972+0000 7fd2df8fd700 1 -- start start 2023-11-04T05:28:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.974+0000 7fd2df8fd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 0x7fd2d80c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.974+0000 7fd2df8fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2d8157910 0x7fd2d80c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.974+0000 7fd2df8fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d815e040 0x7fd2d813f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:49.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.974+0000 7fd2df8fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80c9920 con 0x7fd2d8157910 2023-11-04T05:28:49.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.974+0000 7fd2df8fd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80af800 con 0x7fd2d815e040 2023-11-04T05:28:49.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.974+0000 7fd2df8fd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80c6d30 con 0x7fd2d806b560 2023-11-04T05:28:49.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 0x7fd2d80c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:49.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 0x7fd2d80c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43278/0 (socket says 172.21.15.3:43278) 2023-11-04T05:28:49.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dde9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d815e040 0x7fd2d813f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:49.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 -- 172.21.15.3:0/2145081342 learned_addr learned my addr 172.21.15.3:0/2145081342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:49.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 -- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d815e040 msgr2=0x7fd2d813f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:49.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d815e040 0x7fd2d813f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:49.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 -- 172.21.15.3:0/2145081342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2d8157910 msgr2=0x7fd2d80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:49.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.975+0000 7fd2dd699700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2d8157910 0x7fd2d80c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:49.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.976+0000 7fd2dd699700 1 -- 172.21.15.3:0/2145081342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0009a70 con 0x7fd2d806b560 2023-11-04T05:28:49.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.977+0000 7fd2dd699700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 0x7fd2d80c8c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800c4b0 tx=0x7fd2c800c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.979+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80052d0 con 0x7fd2d806b560 2023-11-04T05:28:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.979+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8005430 con 0x7fd2d806b560 2023-11-04T05:28:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.979+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80038c0 con 0x7fd2d806b560 2023-11-04T05:28:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.979+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d80c9360 con 0x7fd2d806b560 2023-11-04T05:28:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.979+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d80af490 con 0x7fd2d806b560 2023-11-04T05:28:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.981+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2c8003a20 con 0x7fd2d806b560 2023-11-04T05:28:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.982+0000 7fd2cdffb700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd2bc06afd0 0x7fd2bc06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.982+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd2c8088e30 con 0x7fd2d806b560 2023-11-04T05:28:49.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.984+0000 7fd2dce98700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd2bc06afd0 0x7fd2bc06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:49.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.984+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b80052f0 con 0x7fd2d806b560 2023-11-04T05:28:49.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.985+0000 7fd2dce98700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd2bc06afd0 0x7fd2bc06d490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8157390 tx=0x7fd2d40079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:49.990+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd2c8017080 con 0x7fd2d806b560 2023-11-04T05:28:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.001+0000 7fde325dc700 1 Processor -- start 2023-11-04T05:28:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.001+0000 7fde325dc700 1 -- start start 2023-11-04T05:28:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.002+0000 7fde325dc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde2c1579a0 0x7fde2c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.002+0000 7fde325dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde2c15d930 0x7fde2c15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.002+0000 7fde325dc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 0x7fde2c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.002+0000 7fde325dc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2c15e300 con 0x7fde2c15d930 2023-11-04T05:28:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.002+0000 7fde325dc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2c157d80 con 0x7fde2c1579a0 2023-11-04T05:28:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.002+0000 7fde325dc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0729a0 con 0x7fde2c15a340 2023-11-04T05:28:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 0x7fde2c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 0x7fde2c15a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43290/0 (socket says 172.21.15.3:43290) 2023-11-04T05:28:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 -- 172.21.15.3:0/1898215632 learned_addr learned my addr 172.21.15.3:0/1898215632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde2bfff700 1 --2- 172.21.15.3:0/1898215632 >> v2:172.21.15.188:3300/0 conn(0x7fde2c1579a0 0x7fde2c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 -- 172.21.15.3:0/1898215632 >> v2:172.21.15.188:3300/0 conn(0x7fde2c1579a0 msgr2=0x7fde2c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 --2- 172.21.15.3:0/1898215632 >> v2:172.21.15.188:3300/0 conn(0x7fde2c1579a0 0x7fde2c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 -- 172.21.15.3:0/1898215632 >> v2:172.21.15.3:3300/0 conn(0x7fde2c15d930 msgr2=0x7fde2c15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 --2- 172.21.15.3:0/1898215632 >> v2:172.21.15.3:3300/0 conn(0x7fde2c15d930 0x7fde2c15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 -- 172.21.15.3:0/1898215632 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c071900 con 0x7fde2c15a340 2023-11-04T05:28:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde2bfff700 1 --2- 172.21.15.3:0/1898215632 >> v2:172.21.15.188:3300/0 conn(0x7fde2c1579a0 0x7fde2c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.003+0000 7fde30b79700 1 --2- 172.21.15.3:0/1898215632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 0x7fde2c15a7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0cd000 tx=0x7fde20009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9555c93811ab37 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.004+0000 7fde2affd700 1 -- 172.21.15.3:0/1898215632 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20011830 con 0x7fde2c15a340 2023-11-04T05:28:50.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.004+0000 7fde2affd700 1 -- 172.21.15.3:0/1898215632 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20011990 con 0x7fde2c15a340 2023-11-04T05:28:50.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.004+0000 7fde2affd700 1 -- 172.21.15.3:0/1898215632 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20011d20 con 0x7fde2c15a340 2023-11-04T05:28:50.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 -- 172.21.15.3:0/1898215632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 msgr2=0x7fde2c15a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 --2- 172.21.15.3:0/1898215632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 0x7fde2c15a7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0cd000 tx=0x7fde20009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 -- 172.21.15.3:0/1898215632 shutdown_connections 2023-11-04T05:28:50.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 --2- 172.21.15.3:0/1898215632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c15a340 0x7fde2c15a7b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 --2- 172.21.15.3:0/1898215632 >> v2:172.21.15.188:3300/0 conn(0x7fde2c1579a0 0x7fde2c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 --2- 172.21.15.3:0/1898215632 >> v2:172.21.15.3:3300/0 conn(0x7fde2c15d930 0x7fde2c15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 -- 172.21.15.3:0/1898215632 >> 172.21.15.3:0/1898215632 conn(0x7fde2c06b150 msgr2=0x7fde2c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 -- 172.21.15.3:0/1898215632 shutdown_connections 2023-11-04T05:28:50.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.005+0000 7fde325dc700 1 -- 172.21.15.3:0/1898215632 wait complete. 2023-11-04T05:28:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.007+0000 7fde325dc700 1 Processor -- start 2023-11-04T05:28:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.007+0000 7fde325dc700 1 -- start start 2023-11-04T05:28:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.008+0000 7fde325dc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c1579a0 0x7fde2c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.008+0000 7fde325dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 0x7fde2c0c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.008+0000 7fde325dc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c15d930 0x7fde2c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.008+0000 7fde325dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c148d80 con 0x7fde2c15a340 2023-11-04T05:28:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.008+0000 7fde325dc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c149060 con 0x7fde2c15d930 2023-11-04T05:28:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.008+0000 7fde325dc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c149340 con 0x7fde2c1579a0 2023-11-04T05:28:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 0x7fde2c0c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 0x7fde2c0c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52236/0 (socket says 172.21.15.3:52236) 2023-11-04T05:28:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 -- 172.21.15.3:0/1983783915 learned_addr learned my addr 172.21.15.3:0/1983783915 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2bfff700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c1579a0 0x7fde2c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde30b79700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c15d930 0x7fde2c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 -- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c1579a0 msgr2=0x7fde2c148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c1579a0 0x7fde2c148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 -- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c15d930 msgr2=0x7fde2c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c15d930 0x7fde2c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 -- 172.21.15.3:0/1983783915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0094a0 con 0x7fde2c15a340 2023-11-04T05:28:50.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.009+0000 7fde2b7fe700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 0x7fde2c0c5d20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fa0 tx=0x7fde2400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.010+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24010ce0 con 0x7fde2c15a340 2023-11-04T05:28:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.010+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde20009550 con 0x7fde2c15a340 2023-11-04T05:28:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.010+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c149900 con 0x7fde2c15a340 2023-11-04T05:28:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.010+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24010e40 con 0x7fde2c15a340 2023-11-04T05:28:50.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.010+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24018640 con 0x7fde2c15a340 2023-11-04T05:28:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.012+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde240213b0 con 0x7fde2c15a340 2023-11-04T05:28:50.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.012+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c0052f0 con 0x7fde2c15a340 2023-11-04T05:28:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.012+0000 7f755a1ed700 1 Processor -- start 2023-11-04T05:28:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.013+0000 7fde28ff9700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde1806af00 0x7fde1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.013+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde24006e40 con 0x7fde2c15a340 2023-11-04T05:28:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.013+0000 7fde2bfff700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde1806af00 0x7fde1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.013+0000 7f755a1ed700 1 -- start start 2023-11-04T05:28:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f755a1ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75541579a0 0x7f75540cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f755a1ed700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f755415d930 0x7f755415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f755a1ed700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f755415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f755a1ed700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755415e300 con 0x7f75541579a0 2023-11-04T05:28:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f755a1ed700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7554157d80 con 0x7f755415d930 2023-11-04T05:28:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f755a1ed700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75540729a0 con 0x7f755415a340 2023-11-04T05:28:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f755415d930 0x7f755415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f755415d930 0x7f755415ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51332/0 (socket says 172.21.15.3:51332) 2023-11-04T05:28:50.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 -- 172.21.15.3:0/4072986484 learned_addr learned my addr 172.21.15.3:0/4072986484 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7553fff700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f755415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 -- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 msgr2=0x7f755415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f755415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 -- 172.21.15.3:0/4072986484 >> v2:172.21.15.3:3300/0 conn(0x7f75541579a0 msgr2=0x7f75540cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 --2- 172.21.15.3:0/4072986484 >> v2:172.21.15.3:3300/0 conn(0x7f75541579a0 0x7f75540cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7552ffd700 1 -- 172.21.15.3:0/4072986484 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7554071900 con 0x7f755415d930 2023-11-04T05:28:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.014+0000 7f7553fff700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f755415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.015+0000 7fde2bfff700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde1806af00 0x7fde1806d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009cd0 tx=0x7fde1c009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.015+0000 7fdbdc8a5700 1 Processor -- start 2023-11-04T05:28:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.015+0000 7fdbdc8a5700 1 -- start start 2023-11-04T05:28:50.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.016+0000 7f7552ffd700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f755415d930 0x7f755415ddc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75540cd4b0 tx=0x7f754400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ae71946a8c89b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.016+0000 7fdbdc8a5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdbd806b560 0x7fdbd8157910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbdc8a5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd815e040 0x7fdbd80c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbdc8a5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd80729a0 con 0x7fdbd80cc760 2023-11-04T05:28:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbdc8a5700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd8071900 con 0x7fdbd806b560 2023-11-04T05:28:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbdc8a5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8071be0 con 0x7fdbd815e040 2023-11-04T05:28:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdbd80cc760 0x7fdbd815d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57598/0 (socket says 172.21.15.3:57598) 2023-11-04T05:28:50.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd5d9b700 1 -- 172.21.15.3:0/2395118116 learned_addr learned my addr 172.21.15.3:0/2395118116 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:50.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7f75527fc700 1 -- 172.21.15.3:0/4072986484 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544010070 con 0x7f755415d930 2023-11-04T05:28:50.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7f75527fc700 1 -- 172.21.15.3:0/4072986484 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754400da80 con 0x7f755415d930 2023-11-04T05:28:50.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7f75527fc700 1 -- 172.21.15.3:0/4072986484 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754400de10 con 0x7f755415d930 2023-11-04T05:28:50.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd6d9d700 1 --2- 172.21.15.3:0/2395118116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd815e040 0x7fdbd80c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd659c700 1 --2- 172.21.15.3:0/2395118116 >> v2:172.21.15.188:3300/0 conn(0x7fdbd806b560 0x7fdbd8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd6d9d700 1 -- 172.21.15.3:0/2395118116 >> v2:172.21.15.188:3300/0 conn(0x7fdbd806b560 msgr2=0x7fdbd8157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd6d9d700 1 --2- 172.21.15.3:0/2395118116 >> v2:172.21.15.188:3300/0 conn(0x7fdbd806b560 0x7fdbd8157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd6d9d700 1 -- 172.21.15.3:0/2395118116 >> v1:172.21.15.3:6789/0 conn(0x7fdbd80cc760 legacy=0x7fdbd815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.017+0000 7fdbd6d9d700 1 -- 172.21.15.3:0/2395118116 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8071ec0 con 0x7fdbd815e040 2023-11-04T05:28:50.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 -- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f755415d930 msgr2=0x7f755415ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f755415d930 0x7f755415ddc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75540cd4b0 tx=0x7f754400d330 comp rx=0 tx=0).stop 2023-11-04T05:28:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 -- 172.21.15.3:0/4072986484 shutdown_connections 2023-11-04T05:28:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f755415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 --2- 172.21.15.3:0/4072986484 >> v2:172.21.15.3:3300/0 conn(0x7f75541579a0 0x7f75540cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 --2- 172.21.15.3:0/4072986484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f755415d930 0x7f755415ddc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 -- 172.21.15.3:0/4072986484 >> 172.21.15.3:0/4072986484 conn(0x7f755406b150 msgr2=0x7f755406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7fdbd6d9d700 1 --2- 172.21.15.3:0/2395118116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd815e040 0x7fdbd80c9b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009d90 tx=0x7fdbd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0ab477c1745eb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7fdbd559a700 1 -- 172.21.15.3:0/2395118116 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000e070 con 0x7fdbd815e040 2023-11-04T05:28:50.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7fdbd559a700 1 -- 172.21.15.3:0/2395118116 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd000bb20 con 0x7fdbd815e040 2023-11-04T05:28:50.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7fdbd559a700 1 -- 172.21.15.3:0/2395118116 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0012430 con 0x7fdbd815e040 2023-11-04T05:28:50.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 -- 172.21.15.3:0/4072986484 shutdown_connections 2023-11-04T05:28:50.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.018+0000 7f755a1ed700 1 -- 172.21.15.3:0/4072986484 wait complete. 2023-11-04T05:28:50.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.020+0000 7f755a1ed700 1 Processor -- start 2023-11-04T05:28:50.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.021+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/2395118116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd815e040 msgr2=0x7fdbd80c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.021+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/2395118116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd815e040 0x7fdbd80c9b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009d90 tx=0x7fdbd000b330 comp rx=0 tx=0).stop 2023-11-04T05:28:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/2395118116 shutdown_connections 2023-11-04T05:28:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/2395118116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd815e040 0x7fdbd80c9b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/2395118116 >> v2:172.21.15.188:3300/0 conn(0x7fdbd806b560 0x7fdbd8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/2395118116 >> 172.21.15.3:0/2395118116 conn(0x7fdbd806b150 msgr2=0x7fdbd806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/2395118116 shutdown_connections 2023-11-04T05:28:50.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.021+0000 7f755a1ed700 1 -- start start 2023-11-04T05:28:50.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f755a1ed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75541579a0 0x7f7554200cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f755a1ed700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f75541f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f755a1ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 0x7f75541542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f755a1ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541554c0 con 0x7f755415d930 2023-11-04T05:28:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f755a1ed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7554201200 con 0x7f75541579a0 2023-11-04T05:28:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f755a1ed700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7554201810 con 0x7f755415a340 2023-11-04T05:28:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fde240585e0 con 0x7fde2c15a340 2023-11-04T05:28:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/2395118116 wait complete. 2023-11-04T05:28:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 0x7f75541542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 0x7f75541542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52258/0 (socket says 172.21.15.3:52258) 2023-11-04T05:28:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 -- 172.21.15.3:0/2206807686 learned_addr learned my addr 172.21.15.3:0/2206807686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f75537fe700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75541579a0 0x7f7554200cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7552ffd700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f75541f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 -- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 msgr2=0x7f75541f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f75541f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 -- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75541579a0 msgr2=0x7f7554200cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75541579a0 0x7f7554200cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.022+0000 7f7553fff700 1 -- 172.21.15.3:0/2206807686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440085d0 con 0x7f755415d930 2023-11-04T05:28:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.023+0000 7f7553fff700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 0x7f75541542e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f754800c7f0 tx=0x7f754800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011800 con 0x7f755415d930 2023-11-04T05:28:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548011960 con 0x7f755415d930 2023-11-04T05:28:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011d10 con 0x7f755415d930 2023-11-04T05:28:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7fdbdc8a5700 1 Processor -- start 2023-11-04T05:28:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7fdbdc8a5700 1 -- start start 2023-11-04T05:28:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.023+0000 7f0c74cbd700 1 Processor -- start 2023-11-04T05:28:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7f0c74cbd700 1 -- start start 2023-11-04T05:28:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.025+0000 7fdbdc8a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd806b560 0x7fdbd81f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.025+0000 7fdbdc8a5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd80cc760 0x7fdbd80c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.025+0000 7fdbdc8a5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd815e040 0x7fdbd80aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.025+0000 7fdbdc8a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd80a9a50 con 0x7fdbd806b560 2023-11-04T05:28:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.025+0000 7fdbdc8a5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81f2f80 con 0x7fdbd815e040 2023-11-04T05:28:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.025+0000 7fdbdc8a5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81f3260 con 0x7fdbd80cc760 2023-11-04T05:28:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75541f71b0 con 0x7f755415d930 2023-11-04T05:28:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.024+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75540c9050 con 0x7f755415d930 2023-11-04T05:28:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd815e040 0x7fdbd80aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd815e040 0x7fdbd80aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51350/0 (socket says 172.21.15.3:51350) 2023-11-04T05:28:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd6d9d700 1 -- 172.21.15.3:0/909851087 learned_addr learned my addr 172.21.15.3:0/909851087 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd5d9b700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd80cc760 0x7fdbd80c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd5d9b700 1 -- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd815e040 msgr2=0x7fdbd80aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd5d9b700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd815e040 0x7fdbd80aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd5d9b700 1 -- 172.21.15.3:0/909851087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd806b560 msgr2=0x7fdbd81f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd5d9b700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd806b560 0x7fdbd81f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7fdbd5d9b700 1 -- 172.21.15.3:0/909851087 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0009a70 con 0x7fdbd80cc760 2023-11-04T05:28:50.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7f0c74cbd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0c700c6850 0x7f0c700cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7f0c74cbd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c700c4830 con 0x7f0c700cd4d0 2023-11-04T05:28:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7f0c74cbd700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c700cc740 con 0x7f0c700c6c30 2023-11-04T05:28:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7f0c74cbd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c700cc460 con 0x7f0c700c6850 2023-11-04T05:28:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7f0c6ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c700cd4d0 0x7f0c7015a1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57618/0 (socket says 172.21.15.3:57618) 2023-11-04T05:28:50.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.026+0000 7f0c6ed9d700 1 -- 172.21.15.3:0/3345473587 learned_addr learned my addr 172.21.15.3:0/3345473587 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:50.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7fdbd5d9b700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd80cc760 0x7fdbd80c46a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c500 tx=0x7fdbcc00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7f0c6e59c700 1 --2- 172.21.15.3:0/3345473587 >> v2:172.21.15.188:3300/0 conn(0x7f0c700c6850 0x7f0c700cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc005d70 con 0x7fdbd80cc760 2023-11-04T05:28:50.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0070d0 con 0x7fdbd80cc760 2023-11-04T05:28:50.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc007440 con 0x7fdbd80cc760 2023-11-04T05:28:50.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7548022350 con 0x7f755415d930 2023-11-04T05:28:50.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd80a9cc0 con 0x7fdbd80cc760 2023-11-04T05:28:50.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd81f3540 con 0x7fdbd80cc760 2023-11-04T05:28:50.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.027+0000 7f7543fff700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f753806afa0 0x7f753806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.028+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7548089670 con 0x7f755415d930 2023-11-04T05:28:50.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.028+0000 7f75537fe700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f753806afa0 0x7f753806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.029+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60039563 0 0) 0x7f0c700c4830 con 0x7f0c700cd4d0 2023-11-04T05:28:50.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.029+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c580035b0 con 0x7f0c700cd4d0 2023-11-04T05:28:50.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.029+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75300052f0 con 0x7f755415d930 2023-11-04T05:28:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.029+0000 7f75537fe700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f753806afa0 0x7f753806d460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f754c00a400 tx=0x7f754c00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.029+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc0052f0 con 0x7fdbd80cc760 2023-11-04T05:28:50.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.029+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdbcc0075a0 con 0x7fdbd80cc760 2023-11-04T05:28:50.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741371027 0 0) 0x7f0c700cc740 con 0x7f0c700c6c30 2023-11-04T05:28:50.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c700c4830 con 0x7f0c700c6c30 2023-11-04T05:28:50.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7f0c6e59c700 1 -- 172.21.15.3:0/3345473587 >> v1:172.21.15.188:6789/0 conn(0x7f0c700c6c30 legacy=0x7f0c700ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7f0c6e59c700 1 -- 172.21.15.3:0/3345473587 >> v1:172.21.15.3:6789/0 conn(0x7f0c700cd4d0 legacy=0x7f0c7015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7f0c6e59c700 1 -- 172.21.15.3:0/3345473587 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700c41e0 con 0x7f0c700c6850 2023-11-04T05:28:50.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7fdbc6ffd700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbb806af00 0x7fdbb806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbcc088b70 con 0x7fdbd80cc760 2023-11-04T05:28:50.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.030+0000 7fdbd659c700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbb806af00 0x7fdbb806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.032+0000 7f0c6e59c700 1 --2- 172.21.15.3:0/3345473587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700c6850 0x7f0c700cbc10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009820 tx=0x7f0c64009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0d094b87871338 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.032+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64005e30 con 0x7f0c700c6850 2023-11-04T05:28:50.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.032+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c64006030 con 0x7f0c700c6850 2023-11-04T05:28:50.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.032+0000 7f0c6d59a700 1 -- 172.21.15.3:0/3345473587 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c640063c0 con 0x7f0c700c6850 2023-11-04T05:28:50.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.033+0000 7f0c74cbd700 1 -- 172.21.15.3:0/3345473587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700c6850 msgr2=0x7f0c700cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.033+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/3345473587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700c6850 0x7f0c700cbc10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009820 tx=0x7f0c64009b30 comp rx=0 tx=0).stop 2023-11-04T05:28:50.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.033+0000 7f0c74cbd700 1 -- 172.21.15.3:0/3345473587 shutdown_connections 2023-11-04T05:28:50.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.033+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/3345473587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700c6850 0x7f0c700cbc10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.033+0000 7f0c74cbd700 1 -- 172.21.15.3:0/3345473587 >> 172.21.15.3:0/3345473587 conn(0x7f0c700d0560 msgr2=0x7f0c700d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.035+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fdbcc0539e0 con 0x7fdbd80cc760 2023-11-04T05:28:50.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.035+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7548054410 con 0x7f755415d930 2023-11-04T05:28:50.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.037+0000 7fdbd659c700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbb806af00 0x7fdbb806d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800df10 tx=0x7fdbc800f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.037+0000 7f6605b7c700 1 Processor -- start 2023-11-04T05:28:50.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.037+0000 7f6605b7c700 1 -- start start 2023-11-04T05:28:50.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.038+0000 7f6605b7c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001579a0 0x7f66000cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.038+0000 7f6605b7c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f660015d930 0x7f660015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.038+0000 7f6605b7c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6600157d80 con 0x7f660015a340 2023-11-04T05:28:50.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.039+0000 7f6605b7c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660015e2f0 con 0x7f660015d930 2023-11-04T05:28:50.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.039+0000 7f6605b7c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000729a0 con 0x7f66001579a0 2023-11-04T05:28:50.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.039+0000 7f65fffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f660015a340 0x7f660015a750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57626/0 (socket says 172.21.15.3:57626) 2023-11-04T05:28:50.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.039+0000 7f65fffff700 1 -- 172.21.15.3:0/3448883624 learned_addr learned my addr 172.21.15.3:0/3448883624 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:50.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230981243 0 0) 0x7f6600157d80 con 0x7f660015a340 2023-11-04T05:28:50.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65feffd700 1 --2- 172.21.15.3:0/3448883624 >> v2:172.21.15.188:3300/0 conn(0x7f660015d930 0x7f660015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65ff7fe700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001579a0 0x7f66000cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e80035b0 con 0x7f660015a340 2023-11-04T05:28:50.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2989370454 0 0) 0x7f65e80035b0 con 0x7f660015a340 2023-11-04T05:28:50.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001579a0 msgr2=0x7f66000cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001579a0 0x7f66000cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 >> v2:172.21.15.188:3300/0 conn(0x7f660015d930 msgr2=0x7f660015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 --2- 172.21.15.3:0/3448883624 >> v2:172.21.15.188:3300/0 conn(0x7f660015d930 0x7f660015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600071900 con 0x7f660015a340 2023-11-04T05:28:50.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f65f80019f0 con 0x7f660015a340 2023-11-04T05:28:50.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 >> v1:172.21.15.3:6789/0 conn(0x7f660015a340 legacy=0x7f660015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e8003a00 0x7f65e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e8004850 0x7f65e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f65e800fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660015e2f0 con 0x7f65e8005260 2023-11-04T05:28:50.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000729a0 con 0x7f65e8003a00 2023-11-04T05:28:50.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e8010620 con 0x7f65e8004850 2023-11-04T05:28:50.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.040+0000 7f65fffff700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f65e800fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f0c74cbd700 1 -- 172.21.15.3:0/3345473587 shutdown_connections 2023-11-04T05:28:50.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f65fffff700 1 -- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e8004850 msgr2=0x7f65e8004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:50.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f65fffff700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e8004850 0x7f65e8004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f65fffff700 1 -- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e8003a00 msgr2=0x7f65e8003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-04T05:28:50.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f65fffff700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e8003a00 0x7f65e8003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f65fffff700 1 -- 172.21.15.3:0/3448883624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600071900 con 0x7f65e8005260 2023-11-04T05:28:50.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f65fffff700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f65e800fe20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65f800a960 tx=0x7f65e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7caa7933473982 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.042+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8012600 con 0x7f65e8005260 2023-11-04T05:28:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.042+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8012760 con 0x7f65e8005260 2023-11-04T05:28:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.042+0000 7f65fe7fc700 1 -- 172.21.15.3:0/3448883624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8012b10 con 0x7f65e8005260 2023-11-04T05:28:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.041+0000 7f0c74cbd700 1 -- 172.21.15.3:0/3345473587 wait complete. 2023-11-04T05:28:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.043+0000 7f0c74cbd700 1 Processor -- start 2023-11-04T05:28:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.043+0000 7f6605b7c700 1 -- 172.21.15.3:0/3448883624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 msgr2=0x7f65e800fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.043+0000 7f6605b7c700 1 --2- 172.21.15.3:0/3448883624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f65e800fe20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65f800a960 tx=0x7f65e8005880 comp rx=0 tx=0).stop 2023-11-04T05:28:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.043+0000 7f65fffff700 1 -- 172.21.15.3:0/3448883624 reap_dead start 2023-11-04T05:28:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.044+0000 7f6605b7c700 1 -- 172.21.15.3:0/3448883624 shutdown_connections 2023-11-04T05:28:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.044+0000 7f6605b7c700 1 -- 172.21.15.3:0/3448883624 >> 172.21.15.3:0/3448883624 conn(0x7f660006b150 msgr2=0x7f660006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.043+0000 7f0c74cbd700 1 -- start start 2023-11-04T05:28:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c74cbd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700c6850 0x7f0c70153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c74cbd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c700c6c30 0x7f0c701552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c74cbd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700cd4d0 0x7f0c700b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c74cbd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70154070 con 0x7f0c700c6c30 2023-11-04T05:28:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c74cbd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70154e20 con 0x7f0c700cd4d0 2023-11-04T05:28:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c74cbd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700b5120 con 0x7f0c700c6850 2023-11-04T05:28:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c6ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700cd4d0 0x7f0c700b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c6ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700cd4d0 0x7f0c700b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51376/0 (socket says 172.21.15.3:51376) 2023-11-04T05:28:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c6ed9d700 1 -- 172.21.15.3:0/667590977 learned_addr learned my addr 172.21.15.3:0/667590977 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c6e59c700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700c6850 0x7f0c70153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.045+0000 7f0c6dd9b700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c700c6c30 0x7f0c701552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c6e59c700 1 -- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700cd4d0 msgr2=0x7f0c700b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c6e59c700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700cd4d0 0x7f0c700b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c6e59c700 1 -- 172.21.15.3:0/667590977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c700c6c30 msgr2=0x7f0c701552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c6e59c700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c700c6c30 0x7f0c701552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c6e59c700 1 -- 172.21.15.3:0/667590977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64009430 con 0x7f0c700c6850 2023-11-04T05:28:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c6e59c700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700c6850 0x7f0c70153b30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0c64015000 tx=0x7f0c64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64006830 con 0x7f0c700c6850 2023-11-04T05:28:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c64020070 con 0x7f0c700c6850 2023-11-04T05:28:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64006990 con 0x7f0c700c6850 2023-11-04T05:28:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c70154510 con 0x7f0c700c6850 2023-11-04T05:28:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.046+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c701547d0 con 0x7f0c700c6850 2023-11-04T05:28:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.050+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c50005320 con 0x7f0c700c6850 2023-11-04T05:28:50.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.051+0000 7f6605b7c700 1 -- 172.21.15.3:0/3448883624 shutdown_connections 2023-11-04T05:28:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.050+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c64022030 con 0x7f0c700c6850 2023-11-04T05:28:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.051+0000 7f0c5effd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0c5806b590 0x7f0c5806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.051+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0c6408ad50 con 0x7f0c700c6850 2023-11-04T05:28:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.052+0000 7f0c6dd9b700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0c5806b590 0x7f0c5806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.052+0000 7f0c6dd9b700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0c5806b590 0x7f0c5806da50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c700c3500 tx=0x7f0c60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.056+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f0c64055b80 con 0x7f0c700c6850 2023-11-04T05:28:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.062+0000 7f6605b7c700 1 -- 172.21.15.3:0/3448883624 wait complete. 2023-11-04T05:28:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.064+0000 7f6605b7c700 1 Processor -- start 2023-11-04T05:28:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.064+0000 7f6605b7c700 1 -- start start 2023-11-04T05:28:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.065+0000 7f6605b7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f6600155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.065+0000 7f6605b7c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001f7680 0x7f66000c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.065+0000 7f6605b7c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000b0f50 0x7f66001f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.065+0000 7f6605b7c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c79b0 con 0x7f65e8005260 2023-11-04T05:28:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.065+0000 7f6605b7c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c76b0 con 0x7f66000b0f50 2023-11-04T05:28:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.065+0000 7f6605b7c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66001f7f70 con 0x7f66001f7680 2023-11-04T05:28:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f6600155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f6600155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52294/0 (socket says 172.21.15.3:52294) 2023-11-04T05:28:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 -- 172.21.15.3:0/1366780095 learned_addr learned my addr 172.21.15.3:0/1366780095 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65fffff700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000b0f50 0x7f66001f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 -- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001f7680 msgr2=0x7f66000c7e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-04T05:28:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001f7680 0x7f66000c7e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 -- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000b0f50 msgr2=0x7f66001f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000b0f50 0x7f66001f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.066+0000 7f65ff7fe700 1 -- 172.21.15.3:0/1366780095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66001f8250 con 0x7f65e8005260 2023-11-04T05:28:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.067+0000 7f65ff7fe700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f6600155e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65f400c540 tx=0x7f65f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.067+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4011800 con 0x7f65e8005260 2023-11-04T05:28:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.067+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4011960 con 0x7f65e8005260 2023-11-04T05:28:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.067+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4011cf0 con 0x7f65e8005260 2023-11-04T05:28:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.069+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66001f8530 con 0x7f65e8005260 2023-11-04T05:28:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.069+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000c8560 con 0x7f65e8005260 2023-11-04T05:28:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.072+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65f4010350 con 0x7f65e8005260 2023-11-04T05:28:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.072+0000 7f65e7fff700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f65ec06af00 0x7f65ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.072+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f65f4089080 con 0x7f65e8005260 2023-11-04T05:28:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.074+0000 7f65feffd700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f65ec06af00 0x7f65ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.074+0000 7f65feffd700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f65ec06af00 0x7f65ec06d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6600153120 tx=0x7f65f0006be0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.075+0000 7f79030bc700 1 Processor -- start 2023-11-04T05:28:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.075+0000 7f79030bc700 1 -- start start 2023-11-04T05:28:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f79030bc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f79030bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc15e2d0 con 0x7f78fc0c9b40 2023-11-04T05:28:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f79030bc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc157d80 con 0x7f78fc1579a0 2023-11-04T05:28:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f79030bc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0729a0 con 0x7f78fc15d930 2023-11-04T05:28:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66000c8560 con 0x7f65e8005260 2023-11-04T05:28:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f7901659700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f78fc0c9b40 0x7f78fc0c9fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57652/0 (socket says 172.21.15.3:57652) 2023-11-04T05:28:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.077+0000 7f7901659700 1 -- 172.21.15.3:0/266931020 learned_addr learned my addr 172.21.15.3:0/266931020 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.084+0000 7f78fbfff700 1 --2- 172.21.15.3:0/266931020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.087+0000 7f78fb7fe700 1 -- 172.21.15.3:0/266931020 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187640537 0 0) 0x7f78fc15e2d0 con 0x7f78fc0c9b40 2023-11-04T05:28:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.087+0000 7f78fb7fe700 1 -- 172.21.15.3:0/266931020 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e40057c0 con 0x7f78fc0c9b40 2023-11-04T05:28:50.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.087+0000 7f78fbfff700 1 -- 172.21.15.3:0/266931020 >> v1:172.21.15.188:6789/0 conn(0x7f78fc1579a0 legacy=0x7f78fc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.087+0000 7f78fbfff700 1 -- 172.21.15.3:0/266931020 >> v1:172.21.15.3:6789/0 conn(0x7f78fc0c9b40 legacy=0x7f78fc0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.088+0000 7f78fbfff700 1 -- 172.21.15.3:0/266931020 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc071900 con 0x7f78fc15d930 2023-11-04T05:28:50.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.087+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f66000c8560 con 0x7f65e8005260 2023-11-04T05:28:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.088+0000 7f78fbfff700 1 --2- 172.21.15.3:0/266931020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc15dd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78f000aa10 tx=0x7f78f000ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ede9d0cf6b8c6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.088+0000 7f308201d700 1 Processor -- start 2023-11-04T05:28:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.088+0000 7f308201d700 1 -- start start 2023-11-04T05:28:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f308201d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f307c072970 0x7f307c0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f308201d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0cc330 0x7f307c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f308201d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0cd490 con 0x7f307c072d50 2023-11-04T05:28:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f308201d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0cd850 con 0x7f307c072970 2023-11-04T05:28:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f308201d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0cfc40 con 0x7f307c0cc330 2023-11-04T05:28:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f78fb7fe700 1 -- 172.21.15.3:0/266931020 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000cb70 con 0x7f78fc15d930 2023-11-04T05:28:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f78fb7fe700 1 -- 172.21.15.3:0/266931020 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000ccd0 con 0x7f78fc15d930 2023-11-04T05:28:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f78fb7fe700 1 -- 172.21.15.3:0/266931020 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0004250 con 0x7f78fc15d930 2023-11-04T05:28:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f79030bc700 1 -- 172.21.15.3:0/266931020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 msgr2=0x7f78fc15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f79030bc700 1 --2- 172.21.15.3:0/266931020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc15dd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78f000aa10 tx=0x7f78f000ad20 comp rx=0 tx=0).stop 2023-11-04T05:28:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.090+0000 7f307bfff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0cc330 0x7f307c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f79030bc700 1 -- 172.21.15.3:0/266931020 shutdown_connections 2023-11-04T05:28:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f79030bc700 1 --2- 172.21.15.3:0/266931020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc15dd90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f79030bc700 1 -- 172.21.15.3:0/266931020 >> 172.21.15.3:0/266931020 conn(0x7f78fc06b150 msgr2=0x7f78fc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f307bfff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0cc330 0x7f307c0cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43364/0 (socket says 172.21.15.3:43364) 2023-11-04T05:28:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.091+0000 7f307bfff700 1 -- 172.21.15.3:0/1444378288 learned_addr learned my addr 172.21.15.3:0/1444378288 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.092+0000 7f307b7fe700 1 --2- 172.21.15.3:0/1444378288 >> v2:172.21.15.3:3300/0 conn(0x7f307c072970 0x7f307c0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.092+0000 7f307b7fe700 1 -- 172.21.15.3:0/1444378288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0cc330 msgr2=0x7f307c0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.092+0000 7f307b7fe700 1 --2- 172.21.15.3:0/1444378288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0cc330 0x7f307c0cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.092+0000 7f307b7fe700 1 -- 172.21.15.3:0/1444378288 >> v1:172.21.15.3:6789/0 conn(0x7f307c072d50 legacy=0x7f307c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.092+0000 7f307b7fe700 1 -- 172.21.15.3:0/1444378288 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0cff20 con 0x7f307c072970 2023-11-04T05:28:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.093+0000 7f307b7fe700 1 --2- 172.21.15.3:0/1444378288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c072970 0x7f307c0cf380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3074009d90 tx=0x7f307400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f45eff2743ada5 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.093+0000 7f307a7fc700 1 -- 172.21.15.3:0/1444378288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400e070 con 0x7f307c072970 2023-11-04T05:28:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.093+0000 7f307a7fc700 1 -- 172.21.15.3:0/1444378288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074003a20 con 0x7f307c072970 2023-11-04T05:28:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.093+0000 7f307a7fc700 1 -- 172.21.15.3:0/1444378288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074003dd0 con 0x7f307c072970 2023-11-04T05:28:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.094+0000 7f308201d700 1 -- 172.21.15.3:0/1444378288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c072970 msgr2=0x7f307c0cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.094+0000 7f308201d700 1 --2- 172.21.15.3:0/1444378288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c072970 0x7f307c0cf380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3074009d90 tx=0x7f307400b330 comp rx=0 tx=0).stop 2023-11-04T05:28:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.099+0000 7f79030bc700 1 -- 172.21.15.3:0/266931020 shutdown_connections 2023-11-04T05:28:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.099+0000 7f79030bc700 1 -- 172.21.15.3:0/266931020 wait complete. 2023-11-04T05:28:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f308201d700 1 -- 172.21.15.3:0/1444378288 shutdown_connections 2023-11-04T05:28:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f308201d700 1 --2- 172.21.15.3:0/1444378288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c0cc330 0x7f307c0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f308201d700 1 --2- 172.21.15.3:0/1444378288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c072970 0x7f307c0cf380 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f308201d700 1 -- 172.21.15.3:0/1444378288 >> 172.21.15.3:0/1444378288 conn(0x7f307c06b150 msgr2=0x7f307c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f79030bc700 1 Processor -- start 2023-11-04T05:28:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f308201d700 1 -- 172.21.15.3:0/1444378288 shutdown_connections 2023-11-04T05:28:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f79030bc700 1 -- start start 2023-11-04T05:28:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.102+0000 7f79030bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78fc0c9b40 0x7f78fc0c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.102+0000 7f79030bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 0x7f78fc0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.102+0000 7f79030bc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.102+0000 7f79030bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc151730 con 0x7f78fc1579a0 2023-11-04T05:28:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.102+0000 7f79030bc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c71a0 con 0x7f78fc0c9b40 2023-11-04T05:28:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.102+0000 7f79030bc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c7480 con 0x7f78fc15d930 2023-11-04T05:28:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 0x7f78fc0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 0x7f78fc0c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52314/0 (socket says 172.21.15.3:52314) 2023-11-04T05:28:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 -- 172.21.15.3:0/1225141492 learned_addr learned my addr 172.21.15.3:0/1225141492 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f7900e58700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78fc0c9b40 0x7f78fc0c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.101+0000 7f308201d700 1 -- 172.21.15.3:0/1444378288 wait complete. 2023-11-04T05:28:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f308201d700 1 Processor -- start 2023-11-04T05:28:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 -- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 msgr2=0x7f78fc150640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-04T05:28:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc150640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 -- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78fc0c9b40 msgr2=0x7f78fc0c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78fc0c9b40 0x7f78fc0c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f78fbfff700 1 -- 172.21.15.3:0/1225141492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f000a640 con 0x7f78fc1579a0 2023-11-04T05:28:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f78fbfff700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 0x7f78fc0c6b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78f0000c00 tx=0x7f78f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.103+0000 7f308201d700 1 -- start start 2023-11-04T05:28:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f308201d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c072970 0x7f307c0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f308201d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c072d50 0x7f307c0c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f308201d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 0x7f307c203e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f308201d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c02d0 con 0x7f307c0cc330 2023-11-04T05:28:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f308201d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c9a30 con 0x7f307c072970 2023-11-04T05:28:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.104+0000 7f308201d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c9d10 con 0x7f307c072d50 2023-11-04T05:28:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 0x7f307c203e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 0x7f307c203e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52320/0 (socket says 172.21.15.3:52320) 2023-11-04T05:28:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 -- 172.21.15.3:0/2758163509 learned_addr learned my addr 172.21.15.3:0/2758163509 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 -- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c072d50 msgr2=0x7f307c0c8fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-04T05:28:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c072d50 0x7f307c0c8fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 -- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c072970 msgr2=0x7f307c0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c072970 0x7f307c0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 -- 172.21.15.3:0/2758163509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009a70 con 0x7f307c0cc330 2023-11-04T05:28:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.105+0000 7f307bfff700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 0x7f307c203e80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f306c006b20 tx=0x7f306c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.106+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c00c690 con 0x7f307c0cc330 2023-11-04T05:28:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.106+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c00c7f0 con 0x7f307c0cc330 2023-11-04T05:28:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.106+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c00cba0 con 0x7f307c0cc330 2023-11-04T05:28:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.107+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f001d070 con 0x7f78fc1579a0 2023-11-04T05:28:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.107+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000ce40 con 0x7f78fc1579a0 2023-11-04T05:28:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.107+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f00038e0 con 0x7f78fc1579a0 2023-11-04T05:28:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.107+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0c7760 con 0x7f78fc1579a0 2023-11-04T05:28:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.107+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0c7d20 con 0x7f78fc1579a0 2023-11-04T05:28:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.110+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78f0003a40 con 0x7f78fc1579a0 2023-11-04T05:28:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.110+0000 7f78f97fa700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f78e406b3a0 0x7f78e406d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.110+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f78f008a7e0 con 0x7f78fc1579a0 2023-11-04T05:28:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.111+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e8005320 con 0x7f78fc1579a0 2023-11-04T05:28:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.119+0000 7f7900e58700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f78e406b3a0 0x7f78e406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.116+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c0c9ff0 con 0x7f307c0cc330 2023-11-04T05:28:50.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.116+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c0c7780 con 0x7f307c0cc330 2023-11-04T05:28:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.121+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f78f00558e0 con 0x7f78fc1579a0 2023-11-04T05:28:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.121+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30580052f0 con 0x7f307c0cc330 2023-11-04T05:28:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.123+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f306c00cd00 con 0x7f307c0cc330 2023-11-04T05:28:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.124+0000 7f306bfff700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f306406b010 0x7f306406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.124+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f306c089940 con 0x7f307c0cc330 2023-11-04T05:28:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.134+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f306c054660 con 0x7f307c0cc330 2023-11-04T05:28:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.134+0000 7f7900e58700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f78e406b3a0 0x7f78e406d860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f78ec007fd0 tx=0x7f78ec007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.134+0000 7f307b7fe700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f306406b010 0x7f306406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.168+0000 7f307b7fe700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f306406b010 0x7f306406d4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f307400b370 tx=0x7f3074010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.405+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd2b80055a0 con 0x7fd2d806b560 2023-11-04T05:28:50.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.406+0000 7fd2cdffb700 1 -- 172.21.15.3:0/2145081342 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2c8053b80 con 0x7fd2d806b560 2023-11-04T05:28:50.411 INFO:teuthology.orchestra.run.smithi003.stdout:64424509451 2023-11-04T05:28:50.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd2bc06afd0 msgr2=0x7fd2bc06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd2bc06afd0 0x7fd2bc06d490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8157390 tx=0x7fd2d40079c0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 msgr2=0x7fd2d80c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 0x7fd2d80c8c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800c4b0 tx=0x7fd2c800c7c0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 shutdown_connections 2023-11-04T05:28:50.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd2bc06afd0 0x7fd2bc06d490 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d815e040 0x7fd2d813f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d806b560 0x7fd2d80c8c80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 --2- 172.21.15.3:0/2145081342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2d8157910 0x7fd2d80c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.410+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 >> 172.21.15.3:0/2145081342 conn(0x7fd2d806b150 msgr2=0x7fd2d80ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.411+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 shutdown_connections 2023-11-04T05:28:50.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.411+0000 7fd2df8fd700 1 -- 172.21.15.3:0/2145081342 wait complete. 2023-11-04T05:28:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.425+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdbbc005870 con 0x7fdbd80cc760 2023-11-04T05:28:50.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.425+0000 7fdbc6ffd700 1 -- 172.21.15.3:0/909851087 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdbcc017090 con 0x7fdbd80cc760 2023-11-04T05:28:50.429 INFO:teuthology.orchestra.run.smithi003.stdout:60129542156 2023-11-04T05:28:50.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.5 2023-11-04T05:28:50.432 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbb806af00 msgr2=0x7fdbb806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbb806af00 0x7fdbb806d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800df10 tx=0x7fdbc800f040 comp rx=0 tx=0).stop 2023-11-04T05:28:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd80cc760 msgr2=0x7fdbd80c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd80cc760 0x7fdbd80c46a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c500 tx=0x7fdbcc00c810 comp rx=0 tx=0).stop 2023-11-04T05:28:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 shutdown_connections 2023-11-04T05:28:50.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdbb806af00 0x7fdbb806d3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbd815e040 0x7fdbd80aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbd806b560 0x7fdbd81f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 --2- 172.21.15.3:0/909851087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdbd80cc760 0x7fdbd80c46a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.429+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 >> 172.21.15.3:0/909851087 conn(0x7fdbd806b150 msgr2=0x7fdbd80c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.430+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 shutdown_connections 2023-11-04T05:28:50.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.430+0000 7fdbdc8a5700 1 -- 172.21.15.3:0/909851087 wait complete. 2023-11-04T05:28:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.444+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde0c005870 con 0x7fde2c15a340 2023-11-04T05:28:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.445+0000 7fde28ff9700 1 -- 172.21.15.3:0/1983783915 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde2401d020 con 0x7fde2c15a340 2023-11-04T05:28:50.447 INFO:teuthology.orchestra.run.smithi003.stdout:60129542156 2023-11-04T05:28:50.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.3 2023-11-04T05:28:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.450+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde1806af00 msgr2=0x7fde1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.450+0000 7fde325dc700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde1806af00 0x7fde1806d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009cd0 tx=0x7fde1c009210 comp rx=0 tx=0).stop 2023-11-04T05:28:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.450+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 msgr2=0x7fde2c0c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.450+0000 7fde325dc700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 0x7fde2c0c5d20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fa0 tx=0x7fde2400bd30 comp rx=0 tx=0).stop 2023-11-04T05:28:50.454 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 shutdown_connections 2023-11-04T05:28:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde1806af00 0x7fde1806d3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2c15d930 0x7fde2c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde2c1579a0 0x7fde2c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 --2- 172.21.15.3:0/1983783915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde2c15a340 0x7fde2c0c5d20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 >> 172.21.15.3:0/1983783915 conn(0x7fde2c06b150 msgr2=0x7fde2c156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 shutdown_connections 2023-11-04T05:28:50.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7fde325dc700 1 -- 172.21.15.3:0/1983783915 wait complete. 2023-11-04T05:28:50.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.453+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f75300055a0 con 0x7f755415d930 2023-11-04T05:28:50.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.455+0000 7f7543fff700 1 -- 172.21.15.3:0/2206807686 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f754801f090 con 0x7f755415d930 2023-11-04T05:28:50.459 INFO:teuthology.orchestra.run.smithi003.stdout:60129542155 2023-11-04T05:28:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.461+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f753806afa0 msgr2=0x7f753806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.461+0000 7f755a1ed700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f753806afa0 0x7f753806d460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f754c00a400 tx=0x7f754c00d040 comp rx=0 tx=0).stop 2023-11-04T05:28:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.461+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 msgr2=0x7f75541542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.461+0000 7f755a1ed700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 0x7f75541542e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f754800c7f0 tx=0x7f754800cb00 comp rx=0 tx=0).stop 2023-11-04T05:28:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 shutdown_connections 2023-11-04T05:28:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f753806afa0 0x7f753806d460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75541579a0 0x7f7554200cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f755415d930 0x7f75541542e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 --2- 172.21.15.3:0/2206807686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f755415a340 0x7f75541f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 >> 172.21.15.3:0/2206807686 conn(0x7f755406b150 msgr2=0x7f755415a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 shutdown_connections 2023-11-04T05:28:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.462+0000 7f755a1ed700 1 -- 172.21.15.3:0/2206807686 wait complete. 2023-11-04T05:28:50.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.2 2023-11-04T05:28:50.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.469+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0c500055d0 con 0x7f0c700c6850 2023-11-04T05:28:50.472 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.470+0000 7f0c5effd700 1 -- 172.21.15.3:0/667590977 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c640591a0 con 0x7f0c700c6850 2023-11-04T05:28:50.473 INFO:teuthology.orchestra.run.smithi003.stdout:60129542156 2023-11-04T05:28:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.472+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f66000c8560 con 0x7f65e8005260 2023-11-04T05:28:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.472+0000 7f65e7fff700 1 -- 172.21.15.3:0/1366780095 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66000c8560 con 0x7f65e8005260 2023-11-04T05:28:50.475 INFO:teuthology.orchestra.run.smithi003.stdout:64424509452 2023-11-04T05:28:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.474+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0c5806b590 msgr2=0x7f0c5806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.474+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0c5806b590 0x7f0c5806da50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c700c3500 tx=0x7f0c60009040 comp rx=0 tx=0).stop 2023-11-04T05:28:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.474+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700c6850 msgr2=0x7f0c70153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.474+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700c6850 0x7f0c70153b30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0c64015000 tx=0x7f0c64004750 comp rx=0 tx=0).stop 2023-11-04T05:28:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 shutdown_connections 2023-11-04T05:28:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f0c5806b590 0x7f0c5806da50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700cd4d0 0x7f0c700b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700c6850 0x7f0c70153b30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 --2- 172.21.15.3:0/667590977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c700c6c30 0x7f0c701552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 >> 172.21.15.3:0/667590977 conn(0x7f0c700d0560 msgr2=0x7f0c700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 shutdown_connections 2023-11-04T05:28:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f0c74cbd700 1 -- 172.21.15.3:0/667590977 wait complete. 2023-11-04T05:28:50.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f65ec06af00 msgr2=0x7f65ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f65ec06af00 0x7f65ec06d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6600153120 tx=0x7f65f0006be0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 msgr2=0x7f6600155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f6600155e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65f400c540 tx=0x7f65f400c900 comp rx=0 tx=0).stop 2023-11-04T05:28:50.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 shutdown_connections 2023-11-04T05:28:50.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f65ec06af00 0x7f65ec06d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000b0f50 0x7f66001f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8005260 0x7f6600155e60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 --2- 172.21.15.3:0/1366780095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66001f7680 0x7f66000c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 >> 172.21.15.3:0/1366780095 conn(0x7f660006b150 msgr2=0x7f66000ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 shutdown_connections 2023-11-04T05:28:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.475+0000 7f6605b7c700 1 -- 172.21.15.3:0/1366780095 wait complete. 2023-11-04T05:28:50.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.1 2023-11-04T05:28:50.484 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.6 2023-11-04T05:28:50.492 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.0 2023-11-04T05:28:50.496 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.503+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f30580055a0 con 0x7f307c0cc330 2023-11-04T05:28:50.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.503+0000 7f306bfff700 1 -- 172.21.15.3:0/2758163509 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f306c057c80 con 0x7f307c0cc330 2023-11-04T05:28:50.506 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-11-04T05:28:50.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f306406b010 msgr2=0x7f306406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f306406b010 0x7f306406d4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f307400b370 tx=0x7f3074010000 comp rx=0 tx=0).stop 2023-11-04T05:28:50.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 msgr2=0x7f307c203e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 0x7f307c203e80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f306c006b20 tx=0x7f306c00ebe0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 shutdown_connections 2023-11-04T05:28:50.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f306406b010 0x7f306406d4d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f307c072970 0x7f307c0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0cc330 0x7f307c203e80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 --2- 172.21.15.3:0/2758163509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f307c072d50 0x7f307c0c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 >> 172.21.15.3:0/2758163509 conn(0x7f307c06b150 msgr2=0x7f307c0712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 shutdown_connections 2023-11-04T05:28:50.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.506+0000 7f308201d700 1 -- 172.21.15.3:0/2758163509 wait complete. 2023-11-04T05:28:50.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.512+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f78e80058a0 con 0x7f78fc1579a0 2023-11-04T05:28:50.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.512+0000 7f78f97fa700 1 -- 172.21.15.3:0/1225141492 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78f0027020 con 0x7f78fc1579a0 2023-11-04T05:28:50.515 INFO:teuthology.orchestra.run.smithi003.stdout:64424509451 2023-11-04T05:28:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f78e406b3a0 msgr2=0x7f78e406d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f78e406b3a0 0x7f78e406d860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f78ec007fd0 tx=0x7f78ec007ec0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 msgr2=0x7f78fc0c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 0x7f78fc0c6b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78f0000c00 tx=0x7f78f0004750 comp rx=0 tx=0).stop 2023-11-04T05:28:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 shutdown_connections 2023-11-04T05:28:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f78e406b3a0 0x7f78e406d860 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78fc0c9b40 0x7f78fc0c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78fc15d930 0x7f78fc150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 --2- 172.21.15.3:0/1225141492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78fc1579a0 0x7f78fc0c6b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 >> 172.21.15.3:0/1225141492 conn(0x7f78fc06b150 msgr2=0x7f78fc156710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 shutdown_connections 2023-11-04T05:28:50.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.515+0000 7f79030bc700 1 -- 172.21.15.3:0/1225141492 wait complete. 2023-11-04T05:28:50.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2023-11-04T05:28:50.521 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2023-11-04T05:28:50.530 DEBUG:teuthology.parallel:result is None 2023-11-04T05:28:50.530 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-04T05:28:50.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-04T05:28:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.693+0000 7f3537ee2700 1 Processor -- start 2023-11-04T05:28:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.693+0000 7f3537ee2700 1 -- start start 2023-11-04T05:28:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3537ee2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35301526c0 0x7f35300b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3537ee2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3530152aa0 0x7f35300cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3537ee2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353015c250 con 0x7f353015bd70 2023-11-04T05:28:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3537ee2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353015cdd0 con 0x7f3530152aa0 2023-11-04T05:28:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3537ee2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353015c530 con 0x7f35301526c0 2023-11-04T05:28:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353547d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3530152aa0 0x7f35300cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353647f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f353015bd70 0x7f35300cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57680/0 (socket says 172.21.15.3:57680) 2023-11-04T05:28:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353647f700 1 -- 172.21.15.3:0/1530193990 learned_addr learned my addr 172.21.15.3:0/1530193990 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3535c7e700 1 --2- 172.21.15.3:0/1530193990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35301526c0 0x7f35300b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353547d700 1 -- 172.21.15.3:0/1530193990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35301526c0 msgr2=0x7f35300b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353547d700 1 --2- 172.21.15.3:0/1530193990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35301526c0 0x7f35300b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353547d700 1 -- 172.21.15.3:0/1530193990 >> v1:172.21.15.3:6789/0 conn(0x7f353015bd70 legacy=0x7f35300cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f3534c7c700 1 -- 172.21.15.3:0/1530193990 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752584565 0 0) 0x7f353015c250 con 0x7f353015bd70 2023-11-04T05:28:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353547d700 1 -- 172.21.15.3:0/1530193990 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353015c810 con 0x7f3530152aa0 2023-11-04T05:28:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.694+0000 7f353547d700 1 --2- 172.21.15.3:0/1530193990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530152aa0 0x7f35300cde20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3520009840 tx=0x7f3520009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35884a8037c0be6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3534c7c700 1 -- 172.21.15.3:0/1530193990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520011830 con 0x7f3530152aa0 2023-11-04T05:28:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3534c7c700 1 -- 172.21.15.3:0/1530193990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3520011990 con 0x7f3530152aa0 2023-11-04T05:28:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3534c7c700 1 -- 172.21.15.3:0/1530193990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520011d20 con 0x7f3530152aa0 2023-11-04T05:28:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 -- 172.21.15.3:0/1530193990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530152aa0 msgr2=0x7f35300cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 --2- 172.21.15.3:0/1530193990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530152aa0 0x7f35300cde20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3520009840 tx=0x7f3520009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 -- 172.21.15.3:0/1530193990 shutdown_connections 2023-11-04T05:28:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 --2- 172.21.15.3:0/1530193990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35301526c0 0x7f35300b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 --2- 172.21.15.3:0/1530193990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530152aa0 0x7f35300cde20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 -- 172.21.15.3:0/1530193990 >> 172.21.15.3:0/1530193990 conn(0x7f353006ab20 msgr2=0x7f353006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 -- 172.21.15.3:0/1530193990 shutdown_connections 2023-11-04T05:28:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.695+0000 7f3537ee2700 1 -- 172.21.15.3:0/1530193990 wait complete. 2023-11-04T05:28:50.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.696+0000 7f3537ee2700 1 Processor -- start 2023-11-04T05:28:50.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 -- start start 2023-11-04T05:28:50.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 0x7f35300cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f353015bd70 0x7f35300d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35301f2a20 0x7f35300b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353006cb60 con 0x7f35301526c0 2023-11-04T05:28:50.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353006bf50 con 0x7f35301f2a20 2023-11-04T05:28:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3537ee2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353006c230 con 0x7f353015bd70 2023-11-04T05:28:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3535c7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 0x7f35300cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3535c7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 0x7f35300cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52334/0 (socket says 172.21.15.3:52334) 2023-11-04T05:28:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.697+0000 7f3535c7e700 1 -- 172.21.15.3:0/646868288 learned_addr learned my addr 172.21.15.3:0/646868288 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f353547d700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f353015bd70 0x7f35300d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f353647f700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35301f2a20 0x7f35300b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3535c7e700 1 -- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f353015bd70 msgr2=0x7f35300d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3535c7e700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f353015bd70 0x7f35300d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3535c7e700 1 -- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35301f2a20 msgr2=0x7f35300b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3535c7e700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35301f2a20 0x7f35300b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3535c7e700 1 -- 172.21.15.3:0/646868288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200094a0 con 0x7f35301526c0 2023-11-04T05:28:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f353647f700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35301f2a20 0x7f35300b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3535c7e700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 0x7f35300cf170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9b0 tx=0x7f352c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c009710 con 0x7f35301526c0 2023-11-04T05:28:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c00c850 con 0x7f35301526c0 2023-11-04T05:28:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353006c510 con 0x7f35301526c0 2023-11-04T05:28:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00cbc0 con 0x7f35301526c0 2023-11-04T05:28:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.698+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353006db80 con 0x7f35301526c0 2023-11-04T05:28:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.699+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3530074df0 con 0x7f35301526c0 2023-11-04T05:28:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.699+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f352c00cd20 con 0x7f35301526c0 2023-11-04T05:28:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.700+0000 7f35267fc700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f351406af00 0x7f351406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.700+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f352c0899c0 con 0x7f35301526c0 2023-11-04T05:28:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.700+0000 7f353547d700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f351406af00 0x7f351406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.700+0000 7f353547d700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f351406af00 0x7f351406d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35200090b0 tx=0x7f3520008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.704+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f352c054780 con 0x7f35301526c0 2023-11-04T05:28:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.900+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 --> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f353015caf0 con 0x7f351406af00 2023-11-04T05:28:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.910+0000 7f35267fc700 1 -- 172.21.15.3:0/646868288 <== mgr.4099 v2:172.21.15.188:6832/3102759174 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345258 (secure 0 0 0) 0x7f3520011b00 con 0x7f351406af00 2023-11-04T05:28:50.919 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f351406af00 msgr2=0x7f351406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f351406af00 0x7f351406d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35200090b0 tx=0x7f3520008030 comp rx=0 tx=0).stop 2023-11-04T05:28:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 msgr2=0x7f35300cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 0x7f35300cf170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9b0 tx=0x7f352c00ed70 comp rx=0 tx=0).stop 2023-11-04T05:28:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 shutdown_connections 2023-11-04T05:28:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f351406af00 0x7f351406d3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35301f2a20 0x7f35300b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35301526c0 0x7f35300cf170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 --2- 172.21.15.3:0/646868288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f353015bd70 0x7f35300d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 >> 172.21.15.3:0/646868288 conn(0x7f353006ab20 msgr2=0x7f35300b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 shutdown_connections 2023-11-04T05:28:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:50.914+0000 7f3537ee2700 1 -- 172.21.15.3:0/646868288 wait complete. 2023-11-04T05:28:50.925 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-11-04T05:28:50.928 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-11-04T05:28:49.757310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2045232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003676"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835820+0000","last_change":"2023-11-04T05:28:21.455216+0000","last_active":"2023-11-04T05:28:25.835820+0000","last_peered":"2023-11-04T05:28:25.835820+0000","last_clean":"2023-11-04T05:28:25.835820+0000","last_became_active":"2023-11-04T05:28:21.453964+0000","last_became_peered":"2023-11-04T05:28:21.453964+0000","last_unstale":"2023-11-04T05:28:25.835820+0000","last_undegraded":"2023-11-04T05:28:25.835820+0000","last_fullsized":"2023-11-04T05:28:25.835820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858340+0000","last_change":"2023-11-04T05:28:21.458543+0000","last_active":"2023-11-04T05:28:25.858340+0000","last_peered":"2023-11-04T05:28:25.858340+0000","last_clean":"2023-11-04T05:28:25.858340+0000","last_became_active":"2023-11-04T05:28:21.458174+0000","last_became_peered":"2023-11-04T05:28:21.458174+0000","last_unstale":"2023-11-04T05:28:25.858340+0000","last_undegraded":"2023-11-04T05:28:25.858340+0000","last_fullsized":"2023-11-04T05:28:25.858340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842597+0000","last_change":"2023-11-04T05:28:21.457190+0000","last_active":"2023-11-04T05:28:25.842597+0000","last_peered":"2023-11-04T05:28:25.842597+0000","last_clean":"2023-11-04T05:28:25.842597+0000","last_became_active":"2023-11-04T05:28:21.456901+0000","last_became_peered":"2023-11-04T05:28:21.456901+0000","last_unstale":"2023-11-04T05:28:25.842597+0000","last_undegraded":"2023-11-04T05:28:25.842597+0000","last_fullsized":"2023-11-04T05:28:25.842597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:00.273535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840758+0000","last_change":"2023-11-04T05:28:21.464308+0000","last_active":"2023-11-04T05:28:25.840758+0000","last_peered":"2023-11-04T05:28:25.840758+0000","last_clean":"2023-11-04T05:28:25.840758+0000","last_became_active":"2023-11-04T05:28:21.464035+0000","last_became_peered":"2023-11-04T05:28:21.464035+0000","last_unstale":"2023-11-04T05:28:25.840758+0000","last_undegraded":"2023-11-04T05:28:25.840758+0000","last_fullsized":"2023-11-04T05:28:25.840758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:21.443272+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:21.442885+0000","last_became_peered":"2023-11-04T05:28:21.442885+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839412+0000","last_change":"2023-11-04T05:28:21.461562+0000","last_active":"2023-11-04T05:28:25.839412+0000","last_peered":"2023-11-04T05:28:25.839412+0000","last_clean":"2023-11-04T05:28:25.839412+0000","last_became_active":"2023-11-04T05:28:21.461276+0000","last_became_peered":"2023-11-04T05:28:21.461276+0000","last_unstale":"2023-11-04T05:28:25.839412+0000","last_undegraded":"2023-11-04T05:28:25.839412+0000","last_fullsized":"2023-11-04T05:28:25.839412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834303+0000","last_change":"2023-11-04T05:28:21.456621+0000","last_active":"2023-11-04T05:28:25.834303+0000","last_peered":"2023-11-04T05:28:25.834303+0000","last_clean":"2023-11-04T05:28:25.834303+0000","last_became_active":"2023-11-04T05:28:21.456356+0000","last_became_peered":"2023-11-04T05:28:21.456356+0000","last_unstale":"2023-11-04T05:28:25.834303+0000","last_undegraded":"2023-11-04T05:28:25.834303+0000","last_fullsized":"2023-11-04T05:28:25.834303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841746+0000","last_change":"2023-11-04T05:28:21.463624+0000","last_active":"2023-11-04T05:28:25.841746+0000","last_peered":"2023-11-04T05:28:25.841746+0000","last_clean":"2023-11-04T05:28:25.841746+0000","last_became_active":"2023-11-04T05:28:21.463336+0000","last_became_peered":"2023-11-04T05:28:21.463336+0000","last_unstale":"2023-11-04T05:28:25.841746+0000","last_undegraded":"2023-11-04T05:28:25.841746+0000","last_fullsized":"2023-11-04T05:28:25.841746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835730+0000","last_change":"2023-11-04T05:28:21.466992+0000","last_active":"2023-11-04T05:28:25.835730+0000","last_peered":"2023-11-04T05:28:25.835730+0000","last_clean":"2023-11-04T05:28:25.835730+0000","last_became_active":"2023-11-04T05:28:21.466638+0000","last_became_peered":"2023-11-04T05:28:21.466638+0000","last_unstale":"2023-11-04T05:28:25.835730+0000","last_undegraded":"2023-11-04T05:28:25.835730+0000","last_fullsized":"2023-11-04T05:28:25.835730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:01:55.625411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835857+0000","last_change":"2023-11-04T05:28:21.466265+0000","last_active":"2023-11-04T05:28:25.835857+0000","last_peered":"2023-11-04T05:28:25.835857+0000","last_clean":"2023-11-04T05:28:25.835857+0000","last_became_active":"2023-11-04T05:28:21.466080+0000","last_became_peered":"2023-11-04T05:28:21.466080+0000","last_unstale":"2023-11-04T05:28:25.835857+0000","last_undegraded":"2023-11-04T05:28:25.835857+0000","last_fullsized":"2023-11-04T05:28:25.835857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835259+0000","last_change":"2023-11-04T05:28:21.465839+0000","last_active":"2023-11-04T05:28:25.835259+0000","last_peered":"2023-11-04T05:28:25.835259+0000","last_clean":"2023-11-04T05:28:25.835259+0000","last_became_active":"2023-11-04T05:28:21.465571+0000","last_became_peered":"2023-11-04T05:28:21.465571+0000","last_unstale":"2023-11-04T05:28:25.835259+0000","last_undegraded":"2023-11-04T05:28:25.835259+0000","last_fullsized":"2023-11-04T05:28:25.835259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843187+0000","last_change":"2023-11-04T05:28:21.459690+0000","last_active":"2023-11-04T05:28:25.843187+0000","last_peered":"2023-11-04T05:28:25.843187+0000","last_clean":"2023-11-04T05:28:25.843187+0000","last_became_active":"2023-11-04T05:28:21.459406+0000","last_became_peered":"2023-11-04T05:28:21.459406+0000","last_unstale":"2023-11-04T05:28:25.843187+0000","last_undegraded":"2023-11-04T05:28:25.843187+0000","last_fullsized":"2023-11-04T05:28:25.843187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840245+0000","last_change":"2023-11-04T05:28:21.452009+0000","last_active":"2023-11-04T05:28:25.840245+0000","last_peered":"2023-11-04T05:28:25.840245+0000","last_clean":"2023-11-04T05:28:25.840245+0000","last_became_active":"2023-11-04T05:28:21.451701+0000","last_became_peered":"2023-11-04T05:28:21.451701+0000","last_unstale":"2023-11-04T05:28:25.840245+0000","last_undegraded":"2023-11-04T05:28:25.840245+0000","last_fullsized":"2023-11-04T05:28:25.840245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842267+0000","last_change":"2023-11-04T05:28:21.449054+0000","last_active":"2023-11-04T05:28:25.842267+0000","last_peered":"2023-11-04T05:28:25.842267+0000","last_clean":"2023-11-04T05:28:25.842267+0000","last_became_active":"2023-11-04T05:28:21.448593+0000","last_became_peered":"2023-11-04T05:28:21.448593+0000","last_unstale":"2023-11-04T05:28:25.842267+0000","last_undegraded":"2023-11-04T05:28:25.842267+0000","last_fullsized":"2023-11-04T05:28:25.842267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:16:04.673589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857682+0000","last_change":"2023-11-04T05:28:21.459859+0000","last_active":"2023-11-04T05:28:25.857682+0000","last_peered":"2023-11-04T05:28:25.857682+0000","last_clean":"2023-11-04T05:28:25.857682+0000","last_became_active":"2023-11-04T05:28:21.459569+0000","last_became_peered":"2023-11-04T05:28:21.459569+0000","last_unstale":"2023-11-04T05:28:25.857682+0000","last_undegraded":"2023-11-04T05:28:25.857682+0000","last_fullsized":"2023-11-04T05:28:25.857682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839549+0000","last_change":"2023-11-04T05:28:21.461640+0000","last_active":"2023-11-04T05:28:25.839549+0000","last_peered":"2023-11-04T05:28:25.839549+0000","last_clean":"2023-11-04T05:28:25.839549+0000","last_became_active":"2023-11-04T05:28:21.461239+0000","last_became_peered":"2023-11-04T05:28:21.461239+0000","last_unstale":"2023-11-04T05:28:25.839549+0000","last_undegraded":"2023-11-04T05:28:25.839549+0000","last_fullsized":"2023-11-04T05:28:25.839549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841199+0000","last_change":"2023-11-04T05:28:21.463999+0000","last_active":"2023-11-04T05:28:25.841199+0000","last_peered":"2023-11-04T05:28:25.841199+0000","last_clean":"2023-11-04T05:28:25.841199+0000","last_became_active":"2023-11-04T05:28:21.463743+0000","last_became_peered":"2023-11-04T05:28:21.463743+0000","last_unstale":"2023-11-04T05:28:25.841199+0000","last_undegraded":"2023-11-04T05:28:25.841199+0000","last_fullsized":"2023-11-04T05:28:25.841199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846908+0000","last_change":"2023-11-04T05:28:21.454396+0000","last_active":"2023-11-04T05:28:25.846908+0000","last_peered":"2023-11-04T05:28:25.846908+0000","last_clean":"2023-11-04T05:28:25.846908+0000","last_became_active":"2023-11-04T05:28:21.454074+0000","last_became_peered":"2023-11-04T05:28:21.454074+0000","last_unstale":"2023-11-04T05:28:25.846908+0000","last_undegraded":"2023-11-04T05:28:25.846908+0000","last_fullsized":"2023-11-04T05:28:25.846908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835087+0000","last_change":"2023-11-04T05:28:21.441379+0000","last_active":"2023-11-04T05:28:25.835087+0000","last_peered":"2023-11-04T05:28:25.835087+0000","last_clean":"2023-11-04T05:28:25.835087+0000","last_became_active":"2023-11-04T05:28:21.441074+0000","last_became_peered":"2023-11-04T05:28:21.441074+0000","last_unstale":"2023-11-04T05:28:25.835087+0000","last_undegraded":"2023-11-04T05:28:25.835087+0000","last_fullsized":"2023-11-04T05:28:25.835087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220068+0000","last_change":"2023-11-04T05:28:25.856956+0000","last_active":"2023-11-04T05:28:28.220068+0000","last_peered":"2023-11-04T05:28:28.220068+0000","last_clean":"2023-11-04T05:28:28.220068+0000","last_became_active":"2023-11-04T05:28:25.856458+0000","last_became_peered":"2023-11-04T05:28:25.856458+0000","last_unstale":"2023-11-04T05:28:28.220068+0000","last_undegraded":"2023-11-04T05:28:28.220068+0000","last_fullsized":"2023-11-04T05:28:28.220068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:01:17.491615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842941+0000","last_change":"2023-11-04T05:28:20.455408+0000","last_active":"2023-11-04T05:28:25.842941+0000","last_peered":"2023-11-04T05:28:25.842941+0000","last_clean":"2023-11-04T05:28:25.842941+0000","last_became_active":"2023-11-04T05:28:20.455106+0000","last_became_peered":"2023-11-04T05:28:20.455106+0000","last_unstale":"2023-11-04T05:28:25.842941+0000","last_undegraded":"2023-11-04T05:28:25.842941+0000","last_fullsized":"2023-11-04T05:28:25.842941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:09:15.898778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840543+0000","last_change":"2023-11-04T05:28:21.452315+0000","last_active":"2023-11-04T05:28:25.840543+0000","last_peered":"2023-11-04T05:28:25.840543+0000","last_clean":"2023-11-04T05:28:25.840543+0000","last_became_active":"2023-11-04T05:28:21.451728+0000","last_became_peered":"2023-11-04T05:28:21.451728+0000","last_unstale":"2023-11-04T05:28:25.840543+0000","last_undegraded":"2023-11-04T05:28:25.840543+0000","last_fullsized":"2023-11-04T05:28:25.840543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742885+0000","last_change":"2023-11-04T05:28:25.861875+0000","last_active":"2023-11-04T05:28:28.742885+0000","last_peered":"2023-11-04T05:28:28.742885+0000","last_clean":"2023-11-04T05:28:28.742885+0000","last_became_active":"2023-11-04T05:28:25.861458+0000","last_became_peered":"2023-11-04T05:28:25.861458+0000","last_unstale":"2023-11-04T05:28:28.742885+0000","last_undegraded":"2023-11-04T05:28:28.742885+0000","last_fullsized":"2023-11-04T05:28:28.742885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.153647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840648+0000","last_change":"2023-11-04T05:28:20.456506+0000","last_active":"2023-11-04T05:28:25.840648+0000","last_peered":"2023-11-04T05:28:25.840648+0000","last_clean":"2023-11-04T05:28:25.840648+0000","last_became_active":"2023-11-04T05:28:20.456086+0000","last_became_peered":"2023-11-04T05:28:20.456086+0000","last_unstale":"2023-11-04T05:28:25.840648+0000","last_undegraded":"2023-11-04T05:28:25.840648+0000","last_fullsized":"2023-11-04T05:28:25.840648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842845+0000","last_change":"2023-11-04T05:28:21.464433+0000","last_active":"2023-11-04T05:28:25.842845+0000","last_peered":"2023-11-04T05:28:25.842845+0000","last_clean":"2023-11-04T05:28:25.842845+0000","last_became_active":"2023-11-04T05:28:21.464133+0000","last_became_peered":"2023-11-04T05:28:21.464133+0000","last_unstale":"2023-11-04T05:28:25.842845+0000","last_undegraded":"2023-11-04T05:28:25.842845+0000","last_fullsized":"2023-11-04T05:28:25.842845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:26:07.829857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743096+0000","last_change":"2023-11-04T05:28:25.858693+0000","last_active":"2023-11-04T05:28:28.743096+0000","last_peered":"2023-11-04T05:28:28.743096+0000","last_clean":"2023-11-04T05:28:28.743096+0000","last_became_active":"2023-11-04T05:28:25.858151+0000","last_became_peered":"2023-11-04T05:28:25.858151+0000","last_unstale":"2023-11-04T05:28:28.743096+0000","last_undegraded":"2023-11-04T05:28:28.743096+0000","last_fullsized":"2023-11-04T05:28:28.743096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:33:57.728742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:20.448994+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:20.448722+0000","last_became_peered":"2023-11-04T05:28:20.448722+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839002+0000","last_change":"2023-11-04T05:28:21.464488+0000","last_active":"2023-11-04T05:28:25.839002+0000","last_peered":"2023-11-04T05:28:25.839002+0000","last_clean":"2023-11-04T05:28:25.839002+0000","last_became_active":"2023-11-04T05:28:21.462679+0000","last_became_peered":"2023-11-04T05:28:21.462679+0000","last_unstale":"2023-11-04T05:28:25.839002+0000","last_undegraded":"2023-11-04T05:28:25.839002+0000","last_fullsized":"2023-11-04T05:28:25.839002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743030+0000","last_change":"2023-11-04T05:28:25.854516+0000","last_active":"2023-11-04T05:28:28.743030+0000","last_peered":"2023-11-04T05:28:28.743030+0000","last_clean":"2023-11-04T05:28:28.743030+0000","last_became_active":"2023-11-04T05:28:25.854162+0000","last_became_peered":"2023-11-04T05:28:25.854162+0000","last_unstale":"2023-11-04T05:28:28.743030+0000","last_undegraded":"2023-11-04T05:28:28.743030+0000","last_fullsized":"2023-11-04T05:28:28.743030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:06.645428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834998+0000","last_change":"2023-11-04T05:28:20.452770+0000","last_active":"2023-11-04T05:28:25.834998+0000","last_peered":"2023-11-04T05:28:25.834998+0000","last_clean":"2023-11-04T05:28:25.834998+0000","last_became_active":"2023-11-04T05:28:20.452404+0000","last_became_peered":"2023-11-04T05:28:20.452404+0000","last_unstale":"2023-11-04T05:28:25.834998+0000","last_undegraded":"2023-11-04T05:28:25.834998+0000","last_fullsized":"2023-11-04T05:28:25.834998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842144+0000","last_change":"2023-11-04T05:28:21.463294+0000","last_active":"2023-11-04T05:28:25.842144+0000","last_peered":"2023-11-04T05:28:25.842144+0000","last_clean":"2023-11-04T05:28:25.842144+0000","last_became_active":"2023-11-04T05:28:21.463028+0000","last_became_peered":"2023-11-04T05:28:21.463028+0000","last_unstale":"2023-11-04T05:28:25.842144+0000","last_undegraded":"2023-11-04T05:28:25.842144+0000","last_fullsized":"2023-11-04T05:28:25.842144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220306+0000","last_change":"2023-11-04T05:28:25.864434+0000","last_active":"2023-11-04T05:28:28.220306+0000","last_peered":"2023-11-04T05:28:28.220306+0000","last_clean":"2023-11-04T05:28:28.220306+0000","last_became_active":"2023-11-04T05:28:25.863978+0000","last_became_peered":"2023-11-04T05:28:25.863978+0000","last_unstale":"2023-11-04T05:28:28.220306+0000","last_undegraded":"2023-11-04T05:28:28.220306+0000","last_fullsized":"2023-11-04T05:28:28.220306+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:22:02.298285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840436+0000","last_change":"2023-11-04T05:28:20.451845+0000","last_active":"2023-11-04T05:28:25.840436+0000","last_peered":"2023-11-04T05:28:25.840436+0000","last_clean":"2023-11-04T05:28:25.840436+0000","last_became_active":"2023-11-04T05:28:20.451350+0000","last_became_peered":"2023-11-04T05:28:20.451350+0000","last_unstale":"2023-11-04T05:28:25.840436+0000","last_undegraded":"2023-11-04T05:28:25.840436+0000","last_fullsized":"2023-11-04T05:28:25.840436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.837664+0000","last_change":"2023-11-04T05:28:21.454758+0000","last_active":"2023-11-04T05:28:25.837664+0000","last_peered":"2023-11-04T05:28:25.837664+0000","last_clean":"2023-11-04T05:28:25.837664+0000","last_became_active":"2023-11-04T05:28:21.449696+0000","last_became_peered":"2023-11-04T05:28:21.449696+0000","last_unstale":"2023-11-04T05:28:25.837664+0000","last_undegraded":"2023-11-04T05:28:25.837664+0000","last_fullsized":"2023-11-04T05:28:25.837664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:13:54.163937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.177060+0000","last_change":"2023-11-04T05:28:25.869885+0000","last_active":"2023-11-04T05:28:28.177060+0000","last_peered":"2023-11-04T05:28:28.177060+0000","last_clean":"2023-11-04T05:28:28.177060+0000","last_became_active":"2023-11-04T05:28:25.869295+0000","last_became_peered":"2023-11-04T05:28:25.869295+0000","last_unstale":"2023-11-04T05:28:28.177060+0000","last_undegraded":"2023-11-04T05:28:28.177060+0000","last_fullsized":"2023-11-04T05:28:28.177060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:02:41.884046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834648+0000","last_change":"2023-11-04T05:28:20.445804+0000","last_active":"2023-11-04T05:28:25.834648+0000","last_peered":"2023-11-04T05:28:25.834648+0000","last_clean":"2023-11-04T05:28:25.834648+0000","last_became_active":"2023-11-04T05:28:20.445476+0000","last_became_peered":"2023-11-04T05:28:20.445476+0000","last_unstale":"2023-11-04T05:28:25.834648+0000","last_undegraded":"2023-11-04T05:28:25.834648+0000","last_fullsized":"2023-11-04T05:28:25.834648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:22:03.756669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842664+0000","last_change":"2023-11-04T05:28:21.460996+0000","last_active":"2023-11-04T05:28:25.842664+0000","last_peered":"2023-11-04T05:28:25.842664+0000","last_clean":"2023-11-04T05:28:25.842664+0000","last_became_active":"2023-11-04T05:28:21.460451+0000","last_became_peered":"2023-11-04T05:28:21.460451+0000","last_unstale":"2023-11-04T05:28:25.842664+0000","last_undegraded":"2023-11-04T05:28:25.842664+0000","last_fullsized":"2023-11-04T05:28:25.842664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:02.249647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838965+0000","last_change":"2023-11-04T05:28:25.863990+0000","last_active":"2023-11-04T05:28:26.838965+0000","last_peered":"2023-11-04T05:28:26.838965+0000","last_clean":"2023-11-04T05:28:26.838965+0000","last_became_active":"2023-11-04T05:28:25.863587+0000","last_became_peered":"2023-11-04T05:28:25.863587+0000","last_unstale":"2023-11-04T05:28:26.838965+0000","last_undegraded":"2023-11-04T05:28:26.838965+0000","last_fullsized":"2023-11-04T05:28:26.838965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841913+0000","last_change":"2023-11-04T05:28:20.448359+0000","last_active":"2023-11-04T05:28:25.841913+0000","last_peered":"2023-11-04T05:28:25.841913+0000","last_clean":"2023-11-04T05:28:25.841913+0000","last_became_active":"2023-11-04T05:28:20.448037+0000","last_became_peered":"2023-11-04T05:28:20.448037+0000","last_unstale":"2023-11-04T05:28:25.841913+0000","last_undegraded":"2023-11-04T05:28:25.841913+0000","last_fullsized":"2023-11-04T05:28:25.841913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:02:08.721929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839909+0000","last_change":"2023-11-04T05:28:21.459362+0000","last_active":"2023-11-04T05:28:25.839909+0000","last_peered":"2023-11-04T05:28:25.839909+0000","last_clean":"2023-11-04T05:28:25.839909+0000","last_became_active":"2023-11-04T05:28:21.458878+0000","last_became_peered":"2023-11-04T05:28:21.458878+0000","last_unstale":"2023-11-04T05:28:25.839909+0000","last_undegraded":"2023-11-04T05:28:25.839909+0000","last_fullsized":"2023-11-04T05:28:25.839909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919977+0000","last_change":"2023-11-04T05:28:25.865090+0000","last_active":"2023-11-04T05:28:27.919977+0000","last_peered":"2023-11-04T05:28:27.919977+0000","last_clean":"2023-11-04T05:28:27.919977+0000","last_became_active":"2023-11-04T05:28:25.864492+0000","last_became_peered":"2023-11-04T05:28:25.864492+0000","last_unstale":"2023-11-04T05:28:27.919977+0000","last_undegraded":"2023-11-04T05:28:27.919977+0000","last_fullsized":"2023-11-04T05:28:27.919977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:52:12.074596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839654+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.839654+0000","last_peered":"2023-11-04T05:28:25.839654+0000","last_clean":"2023-11-04T05:28:25.839654+0000","last_became_active":"2023-11-04T05:28:20.451322+0000","last_became_peered":"2023-11-04T05:28:20.451322+0000","last_unstale":"2023-11-04T05:28:25.839654+0000","last_undegraded":"2023-11-04T05:28:25.839654+0000","last_fullsized":"2023-11-04T05:28:25.839654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:16.770342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857938+0000","last_change":"2023-11-04T05:28:21.446085+0000","last_active":"2023-11-04T05:28:25.857938+0000","last_peered":"2023-11-04T05:28:25.857938+0000","last_clean":"2023-11-04T05:28:25.857938+0000","last_became_active":"2023-11-04T05:28:21.445831+0000","last_became_peered":"2023-11-04T05:28:21.445831+0000","last_unstale":"2023-11-04T05:28:25.857938+0000","last_undegraded":"2023-11-04T05:28:25.857938+0000","last_fullsized":"2023-11-04T05:28:25.857938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219926+0000","last_change":"2023-11-04T05:28:25.853036+0000","last_active":"2023-11-04T05:28:28.219926+0000","last_peered":"2023-11-04T05:28:28.219926+0000","last_clean":"2023-11-04T05:28:28.219926+0000","last_became_active":"2023-11-04T05:28:25.852217+0000","last_became_peered":"2023-11-04T05:28:25.852217+0000","last_unstale":"2023-11-04T05:28:28.219926+0000","last_undegraded":"2023-11-04T05:28:28.219926+0000","last_fullsized":"2023-11-04T05:28:28.219926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835077+0000","last_change":"2023-11-04T05:28:20.448588+0000","last_active":"2023-11-04T05:28:25.835077+0000","last_peered":"2023-11-04T05:28:25.835077+0000","last_clean":"2023-11-04T05:28:25.835077+0000","last_became_active":"2023-11-04T05:28:20.448330+0000","last_became_peered":"2023-11-04T05:28:20.448330+0000","last_unstale":"2023-11-04T05:28:25.835077+0000","last_undegraded":"2023-11-04T05:28:25.835077+0000","last_fullsized":"2023-11-04T05:28:25.835077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841018+0000","last_change":"2023-11-04T05:28:21.462557+0000","last_active":"2023-11-04T05:28:25.841018+0000","last_peered":"2023-11-04T05:28:25.841018+0000","last_clean":"2023-11-04T05:28:25.841018+0000","last_became_active":"2023-11-04T05:28:21.462247+0000","last_became_peered":"2023-11-04T05:28:21.462247+0000","last_unstale":"2023-11-04T05:28:25.841018+0000","last_undegraded":"2023-11-04T05:28:25.841018+0000","last_fullsized":"2023-11-04T05:28:25.841018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219973+0000","last_change":"2023-11-04T05:28:25.860094+0000","last_active":"2023-11-04T05:28:28.219973+0000","last_peered":"2023-11-04T05:28:28.219973+0000","last_clean":"2023-11-04T05:28:28.219973+0000","last_became_active":"2023-11-04T05:28:25.859164+0000","last_became_peered":"2023-11-04T05:28:25.859164+0000","last_unstale":"2023-11-04T05:28:28.219973+0000","last_undegraded":"2023-11-04T05:28:28.219973+0000","last_fullsized":"2023-11-04T05:28:28.219973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840932+0000","last_change":"2023-11-04T05:28:20.451529+0000","last_active":"2023-11-04T05:28:25.840932+0000","last_peered":"2023-11-04T05:28:25.840932+0000","last_clean":"2023-11-04T05:28:25.840932+0000","last_became_active":"2023-11-04T05:28:20.451236+0000","last_became_peered":"2023-11-04T05:28:20.451236+0000","last_unstale":"2023-11-04T05:28:25.840932+0000","last_undegraded":"2023-11-04T05:28:25.840932+0000","last_fullsized":"2023-11-04T05:28:25.840932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843021+0000","last_change":"2023-11-04T05:28:21.464721+0000","last_active":"2023-11-04T05:28:25.843021+0000","last_peered":"2023-11-04T05:28:25.843021+0000","last_clean":"2023-11-04T05:28:25.843021+0000","last_became_active":"2023-11-04T05:28:21.464403+0000","last_became_peered":"2023-11-04T05:28:21.464403+0000","last_unstale":"2023-11-04T05:28:25.843021+0000","last_undegraded":"2023-11-04T05:28:25.843021+0000","last_fullsized":"2023-11-04T05:28:25.843021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:40.757866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743335+0000","last_change":"2023-11-04T05:28:25.858588+0000","last_active":"2023-11-04T05:28:28.743335+0000","last_peered":"2023-11-04T05:28:28.743335+0000","last_clean":"2023-11-04T05:28:28.743335+0000","last_became_active":"2023-11-04T05:28:25.858202+0000","last_became_peered":"2023-11-04T05:28:25.858202+0000","last_unstale":"2023-11-04T05:28:28.743335+0000","last_undegraded":"2023-11-04T05:28:28.743335+0000","last_fullsized":"2023-11-04T05:28:28.743335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:36:16.890049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858629+0000","last_change":"2023-11-04T05:28:20.451067+0000","last_active":"2023-11-04T05:28:25.858629+0000","last_peered":"2023-11-04T05:28:25.858629+0000","last_clean":"2023-11-04T05:28:25.858629+0000","last_became_active":"2023-11-04T05:28:20.450732+0000","last_became_peered":"2023-11-04T05:28:20.450732+0000","last_unstale":"2023-11-04T05:28:25.858629+0000","last_undegraded":"2023-11-04T05:28:25.858629+0000","last_fullsized":"2023-11-04T05:28:25.858629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843093+0000","last_change":"2023-11-04T05:28:21.457579+0000","last_active":"2023-11-04T05:28:25.843093+0000","last_peered":"2023-11-04T05:28:25.843093+0000","last_clean":"2023-11-04T05:28:25.843093+0000","last_became_active":"2023-11-04T05:28:21.457314+0000","last_became_peered":"2023-11-04T05:28:21.457314+0000","last_unstale":"2023-11-04T05:28:25.843093+0000","last_undegraded":"2023-11-04T05:28:25.843093+0000","last_fullsized":"2023-11-04T05:28:25.843093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832422+0000","last_change":"2023-11-04T05:28:25.863159+0000","last_active":"2023-11-04T05:28:28.832422+0000","last_peered":"2023-11-04T05:28:28.832422+0000","last_clean":"2023-11-04T05:28:28.832422+0000","last_became_active":"2023-11-04T05:28:25.862645+0000","last_became_peered":"2023-11-04T05:28:25.862645+0000","last_unstale":"2023-11-04T05:28:28.832422+0000","last_undegraded":"2023-11-04T05:28:28.832422+0000","last_fullsized":"2023-11-04T05:28:28.832422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:41.867932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:31.743983+0000","last_change":"2023-11-04T05:28:20.449394+0000","last_active":"2023-11-04T05:28:31.743983+0000","last_peered":"2023-11-04T05:28:31.743983+0000","last_clean":"2023-11-04T05:28:31.743983+0000","last_became_active":"2023-11-04T05:28:20.449060+0000","last_became_peered":"2023-11-04T05:28:20.449060+0000","last_unstale":"2023-11-04T05:28:31.743983+0000","last_undegraded":"2023-11-04T05:28:31.743983+0000","last_fullsized":"2023-11-04T05:28:31.743983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835007+0000","last_change":"2023-11-04T05:28:21.459298+0000","last_active":"2023-11-04T05:28:25.835007+0000","last_peered":"2023-11-04T05:28:25.835007+0000","last_clean":"2023-11-04T05:28:25.835007+0000","last_became_active":"2023-11-04T05:28:21.459046+0000","last_became_peered":"2023-11-04T05:28:21.459046+0000","last_unstale":"2023-11-04T05:28:25.835007+0000","last_undegraded":"2023-11-04T05:28:25.835007+0000","last_fullsized":"2023-11-04T05:28:25.835007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836097+0000","last_change":"2023-11-04T05:28:25.853324+0000","last_active":"2023-11-04T05:28:26.836097+0000","last_peered":"2023-11-04T05:28:26.836097+0000","last_clean":"2023-11-04T05:28:26.836097+0000","last_became_active":"2023-11-04T05:28:25.852913+0000","last_became_peered":"2023-11-04T05:28:25.852913+0000","last_unstale":"2023-11-04T05:28:26.836097+0000","last_undegraded":"2023-11-04T05:28:26.836097+0000","last_fullsized":"2023-11-04T05:28:26.836097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842933+0000","last_change":"2023-11-04T05:28:20.451652+0000","last_active":"2023-11-04T05:28:25.842933+0000","last_peered":"2023-11-04T05:28:25.842933+0000","last_clean":"2023-11-04T05:28:25.842933+0000","last_became_active":"2023-11-04T05:28:20.451415+0000","last_became_peered":"2023-11-04T05:28:20.451415+0000","last_unstale":"2023-11-04T05:28:25.842933+0000","last_undegraded":"2023-11-04T05:28:25.842933+0000","last_fullsized":"2023-11-04T05:28:25.842933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.132028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842574+0000","last_change":"2023-11-04T05:28:21.454210+0000","last_active":"2023-11-04T05:28:25.842574+0000","last_peered":"2023-11-04T05:28:25.842574+0000","last_clean":"2023-11-04T05:28:25.842574+0000","last_became_active":"2023-11-04T05:28:21.453894+0000","last_became_peered":"2023-11-04T05:28:21.453894+0000","last_unstale":"2023-11-04T05:28:25.842574+0000","last_undegraded":"2023-11-04T05:28:25.842574+0000","last_fullsized":"2023-11-04T05:28:25.842574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:28:59.983010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220347+0000","last_change":"2023-11-04T05:28:25.864286+0000","last_active":"2023-11-04T05:28:28.220347+0000","last_peered":"2023-11-04T05:28:28.220347+0000","last_clean":"2023-11-04T05:28:28.220347+0000","last_became_active":"2023-11-04T05:28:25.863831+0000","last_became_peered":"2023-11-04T05:28:25.863831+0000","last_unstale":"2023-11-04T05:28:28.220347+0000","last_undegraded":"2023-11-04T05:28:28.220347+0000","last_fullsized":"2023-11-04T05:28:28.220347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:40:57.121972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839219+0000","last_change":"2023-11-04T05:28:20.456235+0000","last_active":"2023-11-04T05:28:25.839219+0000","last_peered":"2023-11-04T05:28:25.839219+0000","last_clean":"2023-11-04T05:28:25.839219+0000","last_became_active":"2023-11-04T05:28:20.455992+0000","last_became_peered":"2023-11-04T05:28:20.455992+0000","last_unstale":"2023-11-04T05:28:25.839219+0000","last_undegraded":"2023-11-04T05:28:25.839219+0000","last_fullsized":"2023-11-04T05:28:25.839219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834802+0000","last_change":"2023-11-04T05:28:21.460351+0000","last_active":"2023-11-04T05:28:25.834802+0000","last_peered":"2023-11-04T05:28:25.834802+0000","last_clean":"2023-11-04T05:28:25.834802+0000","last_became_active":"2023-11-04T05:28:21.459737+0000","last_became_peered":"2023-11-04T05:28:21.459737+0000","last_unstale":"2023-11-04T05:28:25.834802+0000","last_undegraded":"2023-11-04T05:28:25.834802+0000","last_fullsized":"2023-11-04T05:28:25.834802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220525+0000","last_change":"2023-11-04T05:28:25.868351+0000","last_active":"2023-11-04T05:28:28.220525+0000","last_peered":"2023-11-04T05:28:28.220525+0000","last_clean":"2023-11-04T05:28:28.220525+0000","last_became_active":"2023-11-04T05:28:25.867909+0000","last_became_peered":"2023-11-04T05:28:25.867909+0000","last_unstale":"2023-11-04T05:28:28.220525+0000","last_undegraded":"2023-11-04T05:28:28.220525+0000","last_fullsized":"2023-11-04T05:28:28.220525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:08:06.813956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833857+0000","last_change":"2023-11-04T05:28:20.447033+0000","last_active":"2023-11-04T05:28:25.833857+0000","last_peered":"2023-11-04T05:28:25.833857+0000","last_clean":"2023-11-04T05:28:25.833857+0000","last_became_active":"2023-11-04T05:28:20.446767+0000","last_became_peered":"2023-11-04T05:28:20.446767+0000","last_unstale":"2023-11-04T05:28:25.833857+0000","last_undegraded":"2023-11-04T05:28:25.833857+0000","last_fullsized":"2023-11-04T05:28:25.833857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836084+0000","last_change":"2023-11-04T05:28:21.457261+0000","last_active":"2023-11-04T05:28:25.836084+0000","last_peered":"2023-11-04T05:28:25.836084+0000","last_clean":"2023-11-04T05:28:25.836084+0000","last_became_active":"2023-11-04T05:28:21.456982+0000","last_became_peered":"2023-11-04T05:28:21.456982+0000","last_unstale":"2023-11-04T05:28:25.836084+0000","last_undegraded":"2023-11-04T05:28:25.836084+0000","last_fullsized":"2023-11-04T05:28:25.836084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919838+0000","last_change":"2023-11-04T05:28:25.854585+0000","last_active":"2023-11-04T05:28:27.919838+0000","last_peered":"2023-11-04T05:28:27.919838+0000","last_clean":"2023-11-04T05:28:27.919838+0000","last_became_active":"2023-11-04T05:28:25.854151+0000","last_became_peered":"2023-11-04T05:28:25.854151+0000","last_unstale":"2023-11-04T05:28:27.919838+0000","last_undegraded":"2023-11-04T05:28:27.919838+0000","last_fullsized":"2023-11-04T05:28:27.919838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:05:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.844868+0000","last_change":"2023-11-04T05:28:20.448118+0000","last_active":"2023-11-04T05:28:25.844868+0000","last_peered":"2023-11-04T05:28:25.844868+0000","last_clean":"2023-11-04T05:28:25.844868+0000","last_became_active":"2023-11-04T05:28:20.447780+0000","last_became_peered":"2023-11-04T05:28:20.447780+0000","last_unstale":"2023-11-04T05:28:25.844868+0000","last_undegraded":"2023-11-04T05:28:25.844868+0000","last_fullsized":"2023-11-04T05:28:25.844868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:45:52.143624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834466+0000","last_change":"2023-11-04T05:28:21.458858+0000","last_active":"2023-11-04T05:28:25.834466+0000","last_peered":"2023-11-04T05:28:25.834466+0000","last_clean":"2023-11-04T05:28:25.834466+0000","last_became_active":"2023-11-04T05:28:21.458577+0000","last_became_peered":"2023-11-04T05:28:21.458577+0000","last_unstale":"2023-11-04T05:28:25.834466+0000","last_undegraded":"2023-11-04T05:28:25.834466+0000","last_fullsized":"2023-11-04T05:28:25.834466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742613+0000","last_change":"2023-11-04T05:28:25.865986+0000","last_active":"2023-11-04T05:28:28.742613+0000","last_peered":"2023-11-04T05:28:28.742613+0000","last_clean":"2023-11-04T05:28:28.742613+0000","last_became_active":"2023-11-04T05:28:25.865476+0000","last_became_peered":"2023-11-04T05:28:25.865476+0000","last_unstale":"2023-11-04T05:28:28.742613+0000","last_undegraded":"2023-11-04T05:28:28.742613+0000","last_fullsized":"2023-11-04T05:28:28.742613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:23.545163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842973+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.842973+0000","last_peered":"2023-11-04T05:28:25.842973+0000","last_clean":"2023-11-04T05:28:25.842973+0000","last_became_active":"2023-11-04T05:28:20.451353+0000","last_became_peered":"2023-11-04T05:28:20.451353+0000","last_unstale":"2023-11-04T05:28:25.842973+0000","last_undegraded":"2023-11-04T05:28:25.842973+0000","last_fullsized":"2023-11-04T05:28:25.842973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:11:52.224724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835325+0000","last_change":"2023-11-04T05:28:21.460107+0000","last_active":"2023-11-04T05:28:25.835325+0000","last_peered":"2023-11-04T05:28:25.835325+0000","last_clean":"2023-11-04T05:28:25.835325+0000","last_became_active":"2023-11-04T05:28:21.458229+0000","last_became_peered":"2023-11-04T05:28:21.458229+0000","last_unstale":"2023-11-04T05:28:25.835325+0000","last_undegraded":"2023-11-04T05:28:25.835325+0000","last_fullsized":"2023-11-04T05:28:25.835325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832264+0000","last_change":"2023-11-04T05:28:25.863802+0000","last_active":"2023-11-04T05:28:28.832264+0000","last_peered":"2023-11-04T05:28:28.832264+0000","last_clean":"2023-11-04T05:28:28.832264+0000","last_became_active":"2023-11-04T05:28:25.863268+0000","last_became_peered":"2023-11-04T05:28:25.863268+0000","last_unstale":"2023-11-04T05:28:28.832264+0000","last_undegraded":"2023-11-04T05:28:28.832264+0000","last_fullsized":"2023-11-04T05:28:28.832264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:30.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862531+0000","last_change":"2023-11-04T05:28:20.453776+0000","last_active":"2023-11-04T05:28:25.862531+0000","last_peered":"2023-11-04T05:28:25.862531+0000","last_clean":"2023-11-04T05:28:25.862531+0000","last_became_active":"2023-11-04T05:28:20.453564+0000","last_became_peered":"2023-11-04T05:28:20.453564+0000","last_unstale":"2023-11-04T05:28:25.862531+0000","last_undegraded":"2023-11-04T05:28:25.862531+0000","last_fullsized":"2023-11-04T05:28:25.862531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842239+0000","last_change":"2023-11-04T05:28:21.463249+0000","last_active":"2023-11-04T05:28:25.842239+0000","last_peered":"2023-11-04T05:28:25.842239+0000","last_clean":"2023-11-04T05:28:25.842239+0000","last_became_active":"2023-11-04T05:28:21.462964+0000","last_became_peered":"2023-11-04T05:28:21.462964+0000","last_unstale":"2023-11-04T05:28:25.842239+0000","last_undegraded":"2023-11-04T05:28:25.842239+0000","last_fullsized":"2023-11-04T05:28:25.842239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832100+0000","last_change":"2023-11-04T05:28:25.866264+0000","last_active":"2023-11-04T05:28:28.832100+0000","last_peered":"2023-11-04T05:28:28.832100+0000","last_clean":"2023-11-04T05:28:28.832100+0000","last_became_active":"2023-11-04T05:28:25.865946+0000","last_became_peered":"2023-11-04T05:28:25.865946+0000","last_unstale":"2023-11-04T05:28:28.832100+0000","last_undegraded":"2023-11-04T05:28:28.832100+0000","last_fullsized":"2023-11-04T05:28:28.832100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839118+0000","last_change":"2023-11-04T05:28:20.449195+0000","last_active":"2023-11-04T05:28:25.839118+0000","last_peered":"2023-11-04T05:28:25.839118+0000","last_clean":"2023-11-04T05:28:25.839118+0000","last_became_active":"2023-11-04T05:28:20.448862+0000","last_became_peered":"2023-11-04T05:28:20.448862+0000","last_unstale":"2023-11-04T05:28:25.839118+0000","last_undegraded":"2023-11-04T05:28:25.839118+0000","last_fullsized":"2023-11-04T05:28:25.839118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:32.638827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843378+0000","last_change":"2023-11-04T05:28:21.464603+0000","last_active":"2023-11-04T05:28:25.843378+0000","last_peered":"2023-11-04T05:28:25.843378+0000","last_clean":"2023-11-04T05:28:25.843378+0000","last_became_active":"2023-11-04T05:28:21.464337+0000","last_became_peered":"2023-11-04T05:28:21.464337+0000","last_unstale":"2023-11-04T05:28:25.843378+0000","last_undegraded":"2023-11-04T05:28:25.843378+0000","last_fullsized":"2023-11-04T05:28:25.843378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:44.443386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832325+0000","last_change":"2023-11-04T05:28:25.864109+0000","last_active":"2023-11-04T05:28:28.832325+0000","last_peered":"2023-11-04T05:28:28.832325+0000","last_clean":"2023-11-04T05:28:28.832325+0000","last_became_active":"2023-11-04T05:28:25.863293+0000","last_became_peered":"2023-11-04T05:28:25.863293+0000","last_unstale":"2023-11-04T05:28:28.832325+0000","last_undegraded":"2023-11-04T05:28:28.832325+0000","last_fullsized":"2023-11-04T05:28:28.832325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:13.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857963+0000","last_change":"2023-11-04T05:28:21.448655+0000","last_active":"2023-11-04T05:28:25.857963+0000","last_peered":"2023-11-04T05:28:25.857963+0000","last_clean":"2023-11-04T05:28:25.857963+0000","last_became_active":"2023-11-04T05:28:21.448355+0000","last_became_peered":"2023-11-04T05:28:21.448355+0000","last_unstale":"2023-11-04T05:28:25.857963+0000","last_undegraded":"2023-11-04T05:28:25.857963+0000","last_fullsized":"2023-11-04T05:28:25.857963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845533+0000","last_change":"2023-11-04T05:28:20.453706+0000","last_active":"2023-11-04T05:28:25.845533+0000","last_peered":"2023-11-04T05:28:25.845533+0000","last_clean":"2023-11-04T05:28:25.845533+0000","last_became_active":"2023-11-04T05:28:20.453428+0000","last_became_peered":"2023-11-04T05:28:20.453428+0000","last_unstale":"2023-11-04T05:28:25.845533+0000","last_undegraded":"2023-11-04T05:28:25.845533+0000","last_fullsized":"2023-11-04T05:28:25.845533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:43.438258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838700+0000","last_change":"2023-11-04T05:28:25.852681+0000","last_active":"2023-11-04T05:28:26.838700+0000","last_peered":"2023-11-04T05:28:26.838700+0000","last_clean":"2023-11-04T05:28:26.838700+0000","last_became_active":"2023-11-04T05:28:25.852276+0000","last_became_peered":"2023-11-04T05:28:25.852276+0000","last_unstale":"2023-11-04T05:28:26.838700+0000","last_undegraded":"2023-11-04T05:28:26.838700+0000","last_fullsized":"2023-11-04T05:28:26.838700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:28.005917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840730+0000","last_change":"2023-11-04T05:28:21.466480+0000","last_active":"2023-11-04T05:28:25.840730+0000","last_peered":"2023-11-04T05:28:25.840730+0000","last_clean":"2023-11-04T05:28:25.840730+0000","last_became_active":"2023-11-04T05:28:21.466065+0000","last_became_peered":"2023-11-04T05:28:21.466065+0000","last_unstale":"2023-11-04T05:28:25.840730+0000","last_undegraded":"2023-11-04T05:28:25.840730+0000","last_fullsized":"2023-11-04T05:28:25.840730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.838807+0000","last_change":"2023-11-04T05:28:20.451336+0000","last_active":"2023-11-04T05:28:25.838807+0000","last_peered":"2023-11-04T05:28:25.838807+0000","last_clean":"2023-11-04T05:28:25.838807+0000","last_became_active":"2023-11-04T05:28:20.451000+0000","last_became_peered":"2023-11-04T05:28:20.451000+0000","last_unstale":"2023-11-04T05:28:25.838807+0000","last_undegraded":"2023-11-04T05:28:25.838807+0000","last_fullsized":"2023-11-04T05:28:25.838807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:46.798012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834100+0000","last_change":"2023-11-04T05:28:11.829985+0000","last_active":"2023-11-04T05:28:25.834100+0000","last_peered":"2023-11-04T05:28:25.834100+0000","last_clean":"2023-11-04T05:28:25.834100+0000","last_became_active":"2023-11-04T05:28:11.829705+0000","last_became_peered":"2023-11-04T05:28:11.829705+0000","last_unstale":"2023-11-04T05:28:25.834100+0000","last_undegraded":"2023-11-04T05:28:25.834100+0000","last_fullsized":"2023-11-04T05:28:25.834100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_clean_scrub_stamp":"2023-11-04T05:28:10.378947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:46:15.100605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919902+0000","last_change":"2023-11-04T05:28:25.865855+0000","last_active":"2023-11-04T05:28:27.919902+0000","last_peered":"2023-11-04T05:28:27.919902+0000","last_clean":"2023-11-04T05:28:27.919902+0000","last_became_active":"2023-11-04T05:28:25.865538+0000","last_became_peered":"2023-11-04T05:28:25.865538+0000","last_unstale":"2023-11-04T05:28:27.919902+0000","last_undegraded":"2023-11-04T05:28:27.919902+0000","last_fullsized":"2023-11-04T05:28:27.919902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:24:58.810725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842382+0000","last_change":"2023-11-04T05:28:21.461543+0000","last_active":"2023-11-04T05:28:25.842382+0000","last_peered":"2023-11-04T05:28:25.842382+0000","last_clean":"2023-11-04T05:28:25.842382+0000","last_became_active":"2023-11-04T05:28:21.461195+0000","last_became_peered":"2023-11-04T05:28:21.461195+0000","last_unstale":"2023-11-04T05:28:25.842382+0000","last_undegraded":"2023-11-04T05:28:25.842382+0000","last_fullsized":"2023-11-04T05:28:25.842382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:28:07.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834194+0000","last_change":"2023-11-04T05:28:20.450213+0000","last_active":"2023-11-04T05:28:25.834194+0000","last_peered":"2023-11-04T05:28:25.834194+0000","last_clean":"2023-11-04T05:28:25.834194+0000","last_became_active":"2023-11-04T05:28:20.449957+0000","last_became_peered":"2023-11-04T05:28:20.449957+0000","last_unstale":"2023-11-04T05:28:25.834194+0000","last_undegraded":"2023-11-04T05:28:25.834194+0000","last_fullsized":"2023-11-04T05:28:25.834194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:30:12.799059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919653+0000","last_change":"2023-11-04T05:28:25.861935+0000","last_active":"2023-11-04T05:28:27.919653+0000","last_peered":"2023-11-04T05:28:27.919653+0000","last_clean":"2023-11-04T05:28:27.919653+0000","last_became_active":"2023-11-04T05:28:25.861471+0000","last_became_peered":"2023-11-04T05:28:25.861471+0000","last_unstale":"2023-11-04T05:28:27.919653+0000","last_undegraded":"2023-11-04T05:28:27.919653+0000","last_fullsized":"2023-11-04T05:28:27.919653+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:53:47.409110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834605+0000","last_change":"2023-11-04T05:28:20.455821+0000","last_active":"2023-11-04T05:28:25.834605+0000","last_peered":"2023-11-04T05:28:25.834605+0000","last_clean":"2023-11-04T05:28:25.834605+0000","last_became_active":"2023-11-04T05:28:20.455538+0000","last_became_peered":"2023-11-04T05:28:20.455538+0000","last_unstale":"2023-11-04T05:28:25.834605+0000","last_undegraded":"2023-11-04T05:28:25.834605+0000","last_fullsized":"2023-11-04T05:28:25.834605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862653+0000","last_change":"2023-11-04T05:28:21.446022+0000","last_active":"2023-11-04T05:28:25.862653+0000","last_peered":"2023-11-04T05:28:25.862653+0000","last_clean":"2023-11-04T05:28:25.862653+0000","last_became_active":"2023-11-04T05:28:21.445727+0000","last_became_peered":"2023-11-04T05:28:21.445727+0000","last_unstale":"2023-11-04T05:28:25.862653+0000","last_undegraded":"2023-11-04T05:28:25.862653+0000","last_fullsized":"2023-11-04T05:28:25.862653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219703+0000","last_change":"2023-11-04T05:28:25.860350+0000","last_active":"2023-11-04T05:28:28.219703+0000","last_peered":"2023-11-04T05:28:28.219703+0000","last_clean":"2023-11-04T05:28:28.219703+0000","last_became_active":"2023-11-04T05:28:25.859724+0000","last_became_peered":"2023-11-04T05:28:25.859724+0000","last_unstale":"2023-11-04T05:28:28.219703+0000","last_undegraded":"2023-11-04T05:28:28.219703+0000","last_fullsized":"2023-11-04T05:28:28.219703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:33.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.356335+0000","last_change":"2023-11-04T05:28:20.449071+0000","last_active":"2023-11-04T05:28:26.356335+0000","last_peered":"2023-11-04T05:28:26.356335+0000","last_clean":"2023-11-04T05:28:26.356335+0000","last_became_active":"2023-11-04T05:28:20.448778+0000","last_became_peered":"2023-11-04T05:28:20.448778+0000","last_unstale":"2023-11-04T05:28:26.356335+0000","last_undegraded":"2023-11-04T05:28:26.356335+0000","last_fullsized":"2023-11-04T05:28:26.356335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840260+0000","last_change":"2023-11-04T05:28:21.444332+0000","last_active":"2023-11-04T05:28:25.840260+0000","last_peered":"2023-11-04T05:28:25.840260+0000","last_clean":"2023-11-04T05:28:25.840260+0000","last_became_active":"2023-11-04T05:28:21.444027+0000","last_became_peered":"2023-11-04T05:28:21.444027+0000","last_unstale":"2023-11-04T05:28:25.840260+0000","last_undegraded":"2023-11-04T05:28:25.840260+0000","last_fullsized":"2023-11-04T05:28:25.840260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742687+0000","last_change":"2023-11-04T05:28:25.846982+0000","last_active":"2023-11-04T05:28:28.742687+0000","last_peered":"2023-11-04T05:28:28.742687+0000","last_clean":"2023-11-04T05:28:28.742687+0000","last_became_active":"2023-11-04T05:28:25.846562+0000","last_became_peered":"2023-11-04T05:28:25.846562+0000","last_unstale":"2023-11-04T05:28:28.742687+0000","last_undegraded":"2023-11-04T05:28:28.742687+0000","last_fullsized":"2023-11-04T05:28:28.742687+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:52.840764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834969+0000","last_change":"2023-11-04T05:28:20.447063+0000","last_active":"2023-11-04T05:28:25.834969+0000","last_peered":"2023-11-04T05:28:25.834969+0000","last_clean":"2023-11-04T05:28:25.834969+0000","last_became_active":"2023-11-04T05:28:20.446833+0000","last_became_peered":"2023-11-04T05:28:20.446833+0000","last_unstale":"2023-11-04T05:28:25.834969+0000","last_undegraded":"2023-11-04T05:28:25.834969+0000","last_fullsized":"2023-11-04T05:28:25.834969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842874+0000","last_change":"2023-11-04T05:28:21.456075+0000","last_active":"2023-11-04T05:28:25.842874+0000","last_peered":"2023-11-04T05:28:25.842874+0000","last_clean":"2023-11-04T05:28:25.842874+0000","last_became_active":"2023-11-04T05:28:21.455801+0000","last_became_peered":"2023-11-04T05:28:21.455801+0000","last_unstale":"2023-11-04T05:28:25.842874+0000","last_undegraded":"2023-11-04T05:28:25.842874+0000","last_fullsized":"2023-11-04T05:28:25.842874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:48.444983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743366+0000","last_change":"2023-11-04T05:28:25.864053+0000","last_active":"2023-11-04T05:28:28.743366+0000","last_peered":"2023-11-04T05:28:28.743366+0000","last_clean":"2023-11-04T05:28:28.743366+0000","last_became_active":"2023-11-04T05:28:25.863669+0000","last_became_peered":"2023-11-04T05:28:25.863669+0000","last_unstale":"2023-11-04T05:28:28.743366+0000","last_undegraded":"2023-11-04T05:28:28.743366+0000","last_fullsized":"2023-11-04T05:28:28.743366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:42:30.660108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833998+0000","last_change":"2023-11-04T05:28:20.453827+0000","last_active":"2023-11-04T05:28:25.833998+0000","last_peered":"2023-11-04T05:28:25.833998+0000","last_clean":"2023-11-04T05:28:25.833998+0000","last_became_active":"2023-11-04T05:28:20.453576+0000","last_became_peered":"2023-11-04T05:28:20.453576+0000","last_unstale":"2023-11-04T05:28:25.833998+0000","last_undegraded":"2023-11-04T05:28:25.833998+0000","last_fullsized":"2023-11-04T05:28:25.833998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840329+0000","last_change":"2023-11-04T05:28:21.458086+0000","last_active":"2023-11-04T05:28:25.840329+0000","last_peered":"2023-11-04T05:28:25.840329+0000","last_clean":"2023-11-04T05:28:25.840329+0000","last_became_active":"2023-11-04T05:28:21.457739+0000","last_became_peered":"2023-11-04T05:28:21.457739+0000","last_unstale":"2023-11-04T05:28:25.840329+0000","last_undegraded":"2023-11-04T05:28:25.840329+0000","last_fullsized":"2023-11-04T05:28:25.840329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:45.088645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832505+0000","last_change":"2023-11-04T05:28:25.864371+0000","last_active":"2023-11-04T05:28:28.832505+0000","last_peered":"2023-11-04T05:28:28.832505+0000","last_clean":"2023-11-04T05:28:28.832505+0000","last_became_active":"2023-11-04T05:28:25.863374+0000","last_became_peered":"2023-11-04T05:28:25.863374+0000","last_unstale":"2023-11-04T05:28:28.832505+0000","last_undegraded":"2023-11-04T05:28:28.832505+0000","last_fullsized":"2023-11-04T05:28:28.832505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:58.207114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857793+0000","last_change":"2023-11-04T05:28:20.450039+0000","last_active":"2023-11-04T05:28:25.857793+0000","last_peered":"2023-11-04T05:28:25.857793+0000","last_clean":"2023-11-04T05:28:25.857793+0000","last_became_active":"2023-11-04T05:28:20.449761+0000","last_became_peered":"2023-11-04T05:28:20.449761+0000","last_unstale":"2023-11-04T05:28:25.857793+0000","last_undegraded":"2023-11-04T05:28:25.857793+0000","last_fullsized":"2023-11-04T05:28:25.857793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841634+0000","last_change":"2023-11-04T05:28:21.450153+0000","last_active":"2023-11-04T05:28:25.841634+0000","last_peered":"2023-11-04T05:28:25.841634+0000","last_clean":"2023-11-04T05:28:25.841634+0000","last_became_active":"2023-11-04T05:28:21.449878+0000","last_became_peered":"2023-11-04T05:28:21.449878+0000","last_unstale":"2023-11-04T05:28:25.841634+0000","last_undegraded":"2023-11-04T05:28:25.841634+0000","last_fullsized":"2023-11-04T05:28:25.841634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743302+0000","last_change":"2023-11-04T05:28:25.864439+0000","last_active":"2023-11-04T05:28:28.743302+0000","last_peered":"2023-11-04T05:28:28.743302+0000","last_clean":"2023-11-04T05:28:28.743302+0000","last_became_active":"2023-11-04T05:28:25.863909+0000","last_became_peered":"2023-11-04T05:28:25.863909+0000","last_unstale":"2023-11-04T05:28:28.743302+0000","last_undegraded":"2023-11-04T05:28:28.743302+0000","last_fullsized":"2023-11-04T05:28:28.743302+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:00:50.267120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858628+0000","last_change":"2023-11-04T05:28:20.450297+0000","last_active":"2023-11-04T05:28:25.858628+0000","last_peered":"2023-11-04T05:28:25.858628+0000","last_clean":"2023-11-04T05:28:25.858628+0000","last_became_active":"2023-11-04T05:28:20.450017+0000","last_became_peered":"2023-11-04T05:28:20.450017+0000","last_unstale":"2023-11-04T05:28:25.858628+0000","last_undegraded":"2023-11-04T05:28:25.858628+0000","last_fullsized":"2023-11-04T05:28:25.858628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:14:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840528+0000","last_change":"2023-11-04T05:28:21.457231+0000","last_active":"2023-11-04T05:28:25.840528+0000","last_peered":"2023-11-04T05:28:25.840528+0000","last_clean":"2023-11-04T05:28:25.840528+0000","last_became_active":"2023-11-04T05:28:21.456966+0000","last_became_peered":"2023-11-04T05:28:21.456966+0000","last_unstale":"2023-11-04T05:28:25.840528+0000","last_undegraded":"2023-11-04T05:28:25.840528+0000","last_fullsized":"2023-11-04T05:28:25.840528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743383+0000","last_change":"2023-11-04T05:28:25.864991+0000","last_active":"2023-11-04T05:28:28.743383+0000","last_peered":"2023-11-04T05:28:28.743383+0000","last_clean":"2023-11-04T05:28:28.743383+0000","last_became_active":"2023-11-04T05:28:25.864122+0000","last_became_peered":"2023-11-04T05:28:25.864122+0000","last_unstale":"2023-11-04T05:28:28.743383+0000","last_undegraded":"2023-11-04T05:28:28.743383+0000","last_fullsized":"2023-11-04T05:28:28.743383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:28:31.685109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834888+0000","last_change":"2023-11-04T05:28:20.449101+0000","last_active":"2023-11-04T05:28:25.834888+0000","last_peered":"2023-11-04T05:28:25.834888+0000","last_clean":"2023-11-04T05:28:25.834888+0000","last_became_active":"2023-11-04T05:28:20.448809+0000","last_became_peered":"2023-11-04T05:28:20.448809+0000","last_unstale":"2023-11-04T05:28:25.834888+0000","last_undegraded":"2023-11-04T05:28:25.834888+0000","last_fullsized":"2023-11-04T05:28:25.834888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834773+0000","last_change":"2023-11-04T05:28:21.456943+0000","last_active":"2023-11-04T05:28:25.834773+0000","last_peered":"2023-11-04T05:28:25.834773+0000","last_clean":"2023-11-04T05:28:25.834773+0000","last_became_active":"2023-11-04T05:28:21.456658+0000","last_became_peered":"2023-11-04T05:28:21.456658+0000","last_unstale":"2023-11-04T05:28:25.834773+0000","last_undegraded":"2023-11-04T05:28:25.834773+0000","last_fullsized":"2023-11-04T05:28:25.834773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838509+0000","last_change":"2023-11-04T05:28:25.865380+0000","last_active":"2023-11-04T05:28:26.838509+0000","last_peered":"2023-11-04T05:28:26.838509+0000","last_clean":"2023-11-04T05:28:26.838509+0000","last_became_active":"2023-11-04T05:28:25.865014+0000","last_became_peered":"2023-11-04T05:28:25.865014+0000","last_unstale":"2023-11-04T05:28:26.838509+0000","last_undegraded":"2023-11-04T05:28:26.838509+0000","last_fullsized":"2023-11-04T05:28:26.838509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:08.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846470+0000","last_change":"2023-11-04T05:28:20.451801+0000","last_active":"2023-11-04T05:28:25.846470+0000","last_peered":"2023-11-04T05:28:25.846470+0000","last_clean":"2023-11-04T05:28:25.846470+0000","last_became_active":"2023-11-04T05:28:20.451470+0000","last_became_peered":"2023-11-04T05:28:20.451470+0000","last_unstale":"2023-11-04T05:28:25.846470+0000","last_undegraded":"2023-11-04T05:28:25.846470+0000","last_fullsized":"2023-11-04T05:28:25.846470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:26:53.409369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834801+0000","last_change":"2023-11-04T05:28:21.465901+0000","last_active":"2023-11-04T05:28:25.834801+0000","last_peered":"2023-11-04T05:28:25.834801+0000","last_clean":"2023-11-04T05:28:25.834801+0000","last_became_active":"2023-11-04T05:28:21.465612+0000","last_became_peered":"2023-11-04T05:28:21.465612+0000","last_unstale":"2023-11-04T05:28:25.834801+0000","last_undegraded":"2023-11-04T05:28:25.834801+0000","last_fullsized":"2023-11-04T05:28:25.834801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919553+0000","last_change":"2023-11-04T05:28:25.858664+0000","last_active":"2023-11-04T05:28:27.919553+0000","last_peered":"2023-11-04T05:28:27.919553+0000","last_clean":"2023-11-04T05:28:27.919553+0000","last_became_active":"2023-11-04T05:28:25.858208+0000","last_became_peered":"2023-11-04T05:28:25.858208+0000","last_unstale":"2023-11-04T05:28:27.919553+0000","last_undegraded":"2023-11-04T05:28:27.919553+0000","last_fullsized":"2023-11-04T05:28:27.919553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:41:26.004032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858515+0000","last_change":"2023-11-04T05:28:20.443824+0000","last_active":"2023-11-04T05:28:25.858515+0000","last_peered":"2023-11-04T05:28:25.858515+0000","last_clean":"2023-11-04T05:28:25.858515+0000","last_became_active":"2023-11-04T05:28:20.443479+0000","last_became_peered":"2023-11-04T05:28:20.443479+0000","last_unstale":"2023-11-04T05:28:25.858515+0000","last_undegraded":"2023-11-04T05:28:25.858515+0000","last_fullsized":"2023-11-04T05:28:25.858515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840613+0000","last_change":"2023-11-04T05:28:21.463699+0000","last_active":"2023-11-04T05:28:25.840613+0000","last_peered":"2023-11-04T05:28:25.840613+0000","last_clean":"2023-11-04T05:28:25.840613+0000","last_became_active":"2023-11-04T05:28:21.462308+0000","last_became_peered":"2023-11-04T05:28:21.462308+0000","last_unstale":"2023-11-04T05:28:25.840613+0000","last_undegraded":"2023-11-04T05:28:25.840613+0000","last_fullsized":"2023-11-04T05:28:25.840613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836473+0000","last_change":"2023-11-04T05:28:25.849562+0000","last_active":"2023-11-04T05:28:26.836473+0000","last_peered":"2023-11-04T05:28:26.836473+0000","last_clean":"2023-11-04T05:28:26.836473+0000","last_became_active":"2023-11-04T05:28:25.849142+0000","last_became_peered":"2023-11-04T05:28:25.849142+0000","last_unstale":"2023-11-04T05:28:26.836473+0000","last_undegraded":"2023-11-04T05:28:26.836473+0000","last_fullsized":"2023-11-04T05:28:26.836473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846568+0000","last_change":"2023-11-04T05:28:20.455376+0000","last_active":"2023-11-04T05:28:25.846568+0000","last_peered":"2023-11-04T05:28:25.846568+0000","last_clean":"2023-11-04T05:28:25.846568+0000","last_became_active":"2023-11-04T05:28:20.455069+0000","last_became_peered":"2023-11-04T05:28:20.455069+0000","last_unstale":"2023-11-04T05:28:25.846568+0000","last_undegraded":"2023-11-04T05:28:25.846568+0000","last_fullsized":"2023-11-04T05:28:25.846568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:44.083516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835570+0000","last_change":"2023-11-04T05:28:21.459762+0000","last_active":"2023-11-04T05:28:25.835570+0000","last_peered":"2023-11-04T05:28:25.835570+0000","last_clean":"2023-11-04T05:28:25.835570+0000","last_became_active":"2023-11-04T05:28:21.459488+0000","last_became_peered":"2023-11-04T05:28:21.459488+0000","last_unstale":"2023-11-04T05:28:25.835570+0000","last_undegraded":"2023-11-04T05:28:25.835570+0000","last_fullsized":"2023-11-04T05:28:25.835570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836085+0000","last_change":"2023-11-04T05:28:21.458229+0000","last_active":"2023-11-04T05:28:25.836085+0000","last_peered":"2023-11-04T05:28:25.836085+0000","last_clean":"2023-11-04T05:28:25.836085+0000","last_became_active":"2023-11-04T05:28:21.457918+0000","last_became_peered":"2023-11-04T05:28:21.457918+0000","last_unstale":"2023-11-04T05:28:25.836085+0000","last_undegraded":"2023-11-04T05:28:25.836085+0000","last_fullsized":"2023-11-04T05:28:25.836085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845334+0000","last_change":"2023-11-04T05:28:21.464900+0000","last_active":"2023-11-04T05:28:25.845334+0000","last_peered":"2023-11-04T05:28:25.845334+0000","last_clean":"2023-11-04T05:28:25.845334+0000","last_became_active":"2023-11-04T05:28:21.463859+0000","last_became_peered":"2023-11-04T05:28:21.463859+0000","last_unstale":"2023-11-04T05:28:25.845334+0000","last_undegraded":"2023-11-04T05:28:25.845334+0000","last_fullsized":"2023-11-04T05:28:25.845334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839310+0000","last_change":"2023-11-04T05:28:21.466114+0000","last_active":"2023-11-04T05:28:25.839310+0000","last_peered":"2023-11-04T05:28:25.839310+0000","last_clean":"2023-11-04T05:28:25.839310+0000","last_became_active":"2023-11-04T05:28:21.465547+0000","last_became_peered":"2023-11-04T05:28:21.465547+0000","last_unstale":"2023-11-04T05:28:25.839310+0000","last_undegraded":"2023-11-04T05:28:25.839310+0000","last_fullsized":"2023-11-04T05:28:25.839310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842759+0000","last_change":"2023-11-04T05:28:21.456527+0000","last_active":"2023-11-04T05:28:25.842759+0000","last_peered":"2023-11-04T05:28:25.842759+0000","last_clean":"2023-11-04T05:28:25.842759+0000","last_became_active":"2023-11-04T05:28:21.456157+0000","last_became_peered":"2023-11-04T05:28:21.456157+0000","last_unstale":"2023-11-04T05:28:25.842759+0000","last_undegraded":"2023-11-04T05:28:25.842759+0000","last_fullsized":"2023-11-04T05:28:25.842759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:19.149382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840466+0000","last_change":"2023-11-04T05:28:21.460439+0000","last_active":"2023-11-04T05:28:25.840466+0000","last_peered":"2023-11-04T05:28:25.840466+0000","last_clean":"2023-11-04T05:28:25.840466+0000","last_became_active":"2023-11-04T05:28:21.460146+0000","last_became_peered":"2023-11-04T05:28:21.460146+0000","last_unstale":"2023-11-04T05:28:25.840466+0000","last_undegraded":"2023-11-04T05:28:25.840466+0000","last_fullsized":"2023-11-04T05:28:25.840466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858719+0000","last_change":"2023-11-04T05:28:21.447651+0000","last_active":"2023-11-04T05:28:25.858719+0000","last_peered":"2023-11-04T05:28:25.858719+0000","last_clean":"2023-11-04T05:28:25.858719+0000","last_became_active":"2023-11-04T05:28:21.447381+0000","last_became_peered":"2023-11-04T05:28:21.447381+0000","last_unstale":"2023-11-04T05:28:25.858719+0000","last_undegraded":"2023-11-04T05:28:25.858719+0000","last_fullsized":"2023-11-04T05:28:25.858719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835914+0000","last_change":"2023-11-04T05:28:21.443196+0000","last_active":"2023-11-04T05:28:25.835914+0000","last_peered":"2023-11-04T05:28:25.835914+0000","last_clean":"2023-11-04T05:28:25.835914+0000","last_became_active":"2023-11-04T05:28:21.442860+0000","last_became_peered":"2023-11-04T05:28:21.442860+0000","last_unstale":"2023-11-04T05:28:25.835914+0000","last_undegraded":"2023-11-04T05:28:25.835914+0000","last_fullsized":"2023-11-04T05:28:25.835914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841180+0000","last_change":"2023-11-04T05:28:21.455732+0000","last_active":"2023-11-04T05:28:25.841180+0000","last_peered":"2023-11-04T05:28:25.841180+0000","last_clean":"2023-11-04T05:28:25.841180+0000","last_became_active":"2023-11-04T05:28:21.455434+0000","last_became_peered":"2023-11-04T05:28:21.455434+0000","last_unstale":"2023-11-04T05:28:25.841180+0000","last_undegraded":"2023-11-04T05:28:25.841180+0000","last_fullsized":"2023-11-04T05:28:25.841180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839745+0000","last_change":"2023-11-04T05:28:21.457789+0000","last_active":"2023-11-04T05:28:25.839745+0000","last_peered":"2023-11-04T05:28:25.839745+0000","last_clean":"2023-11-04T05:28:25.839745+0000","last_became_active":"2023-11-04T05:28:21.457521+0000","last_became_peered":"2023-11-04T05:28:21.457521+0000","last_unstale":"2023-11-04T05:28:25.839745+0000","last_undegraded":"2023-11-04T05:28:25.839745+0000","last_fullsized":"2023-11-04T05:28:25.839745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841996+0000","last_change":"2023-11-04T05:28:21.462501+0000","last_active":"2023-11-04T05:28:25.841996+0000","last_peered":"2023-11-04T05:28:25.841996+0000","last_clean":"2023-11-04T05:28:25.841996+0000","last_became_active":"2023-11-04T05:28:21.462147+0000","last_became_peered":"2023-11-04T05:28:21.462147+0000","last_unstale":"2023-11-04T05:28:25.841996+0000","last_undegraded":"2023-11-04T05:28:25.841996+0000","last_fullsized":"2023-11-04T05:28:25.841996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839367+0000","last_change":"2023-11-04T05:28:21.462044+0000","last_active":"2023-11-04T05:28:25.839367+0000","last_peered":"2023-11-04T05:28:25.839367+0000","last_clean":"2023-11-04T05:28:25.839367+0000","last_became_active":"2023-11-04T05:28:21.461816+0000","last_became_peered":"2023-11-04T05:28:21.461816+0000","last_unstale":"2023-11-04T05:28:25.839367+0000","last_undegraded":"2023-11-04T05:28:25.839367+0000","last_fullsized":"2023-11-04T05:28:25.839367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843103+0000","last_change":"2023-11-04T05:28:21.458256+0000","last_active":"2023-11-04T05:28:25.843103+0000","last_peered":"2023-11-04T05:28:25.843103+0000","last_clean":"2023-11-04T05:28:25.843103+0000","last_became_active":"2023-11-04T05:28:21.457944+0000","last_became_peered":"2023-11-04T05:28:21.457944+0000","last_unstale":"2023-11-04T05:28:25.843103+0000","last_undegraded":"2023-11-04T05:28:25.843103+0000","last_fullsized":"2023-11-04T05:28:25.843103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:41:28.397824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834927+0000","last_change":"2023-11-04T05:28:21.451553+0000","last_active":"2023-11-04T05:28:25.834927+0000","last_peered":"2023-11-04T05:28:25.834927+0000","last_clean":"2023-11-04T05:28:25.834927+0000","last_became_active":"2023-11-04T05:28:21.451276+0000","last_became_peered":"2023-11-04T05:28:21.451276+0000","last_unstale":"2023-11-04T05:28:25.834927+0000","last_undegraded":"2023-11-04T05:28:25.834927+0000","last_fullsized":"2023-11-04T05:28:25.834927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":697852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":697908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.958}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.939}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":107028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542156,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":106992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-04T05:28:50.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-04T05:28:51.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.092+0000 7f7507c1f700 1 Processor -- start 2023-11-04T05:28:51.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.092+0000 7f7507c1f700 1 -- start start 2023-11-04T05:28:51.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f7507c1f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f75000728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f7507c1f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7500072df0 con 0x7f750006af00 2023-11-04T05:28:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f7507c1f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75000cda00 con 0x7f750006ab20 2023-11-04T05:28:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f7507c1f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75000cf050 con 0x7f75000cc7a0 2023-11-04T05:28:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f75061bc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f750006af00 0x7f75000d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57694/0 (socket says 172.21.15.3:57694) 2023-11-04T05:28:51.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f75061bc700 1 -- 172.21.15.3:0/1900448393 learned_addr learned my addr 172.21.15.3:0/1900448393 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.093+0000 7f75051ba700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f75000728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563107411 0 0) 0x7f7500072df0 con 0x7f750006af00 2023-11-04T05:28:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec0035b0 con 0x7f750006af00 2023-11-04T05:28:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683908156 0 0) 0x7f75000cda00 con 0x7f750006ab20 2023-11-04T05:28:51.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7500072df0 con 0x7f750006ab20 2023-11-04T05:28:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110836546 0 0) 0x7f74ec0035b0 con 0x7f750006af00 2023-11-04T05:28:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 msgr2=0x7f75000728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f75000728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 >> v1:172.21.15.188:6789/0 conn(0x7f750006ab20 legacy=0x7f75000cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75000cdce0 con 0x7f750006af00 2023-11-04T05:28:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f74f80019f0 con 0x7f750006af00 2023-11-04T05:28:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f75000728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 >> v1:172.21.15.3:6789/0 conn(0x7f750006af00 legacy=0x7f75000d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec003bc0 0x7f74ec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ec004850 0x7f74ec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f74ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75000cf050 con 0x7f74ec004850 2023-11-04T05:28:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec00ba40 con 0x7f74ec003bc0 2023-11-04T05:28:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec00bc00 con 0x7f75000cc7a0 2023-11-04T05:28:51.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ec004850 0x7f74ec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75059bb700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec003bc0 0x7f74ec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75061bc700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f74ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 -- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 msgr2=0x7f74ec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000cc7a0 0x7f74ec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 -- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec003bc0 msgr2=0x7f74ec003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec003bc0 0x7f74ec003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75051ba700 1 -- 172.21.15.3:0/1900448393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc009430 con 0x7f74ec004850 2023-11-04T05:28:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.094+0000 7f75059bb700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec003bc0 0x7f74ec003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f75051ba700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ec004850 0x7f74ec004cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ce0 tx=0x7f74ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe9e050e766b9fd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc005a20 con 0x7f74ec004850 2023-11-04T05:28:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc005b80 con 0x7f74ec004850 2023-11-04T05:28:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f75049b9700 1 -- 172.21.15.3:0/1900448393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00e7e0 con 0x7f74ec004850 2023-11-04T05:28:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f7507c1f700 1 -- 172.21.15.3:0/1900448393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ec004850 msgr2=0x7f74ec004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1900448393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ec004850 0x7f74ec004cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ce0 tx=0x7f74ec005880 comp rx=0 tx=0).stop 2023-11-04T05:28:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f75061bc700 1 -- 172.21.15.3:0/1900448393 reap_dead start 2023-11-04T05:28:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f7507c1f700 1 -- 172.21.15.3:0/1900448393 shutdown_connections 2023-11-04T05:28:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f7507c1f700 1 -- 172.21.15.3:0/1900448393 >> 172.21.15.3:0/1900448393 conn(0x7f750006df90 msgr2=0x7f750006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f7507c1f700 1 -- 172.21.15.3:0/1900448393 shutdown_connections 2023-11-04T05:28:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.095+0000 7f7507c1f700 1 -- 172.21.15.3:0/1900448393 wait complete. 2023-11-04T05:28:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 Processor -- start 2023-11-04T05:28:51.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 -- start start 2023-11-04T05:28:51.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec004850 0x7f7500158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 0x7f75000b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000c9a00 0x7f75000c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75000b5e10 con 0x7f75000b2fd0 2023-11-04T05:28:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75000ca3c0 con 0x7f74ec004850 2023-11-04T05:28:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f7507c1f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75000b5190 con 0x7f75000c9a00 2023-11-04T05:28:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f75051ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 0x7f75000b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f75059bb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec004850 0x7f7500158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.097+0000 7f75051ba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 0x7f75000b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52356/0 (socket says 172.21.15.3:52356) 2023-11-04T05:28:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 -- 172.21.15.3:0/1089526332 learned_addr learned my addr 172.21.15.3:0/1089526332 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75061bc700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000c9a00 0x7f75000c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 -- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000c9a00 msgr2=0x7f75000c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000c9a00 0x7f75000c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 -- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec004850 msgr2=0x7f7500158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec004850 0x7f7500158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 -- 172.21.15.3:0/1089526332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75000b5470 con 0x7f75000b2fd0 2023-11-04T05:28:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f75051ba700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 0x7f75000b3450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00ba10 tx=0x7f74fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00ebf0 con 0x7f75000b2fd0 2023-11-04T05:28:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc007e90 con 0x7f75000b2fd0 2023-11-04T05:28:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75000b5750 con 0x7f75000b2fd0 2023-11-04T05:28:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0036a0 con 0x7f75000b2fd0 2023-11-04T05:28:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.098+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75000b4d40 con 0x7f75000b2fd0 2023-11-04T05:28:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.099+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75000b4f00 con 0x7f75000b2fd0 2023-11-04T05:28:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.099+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74fc023070 con 0x7f75000b2fd0 2023-11-04T05:28:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.100+0000 7f74f67fc700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f74e406af00 0x7f74e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.100+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74fc08adf0 con 0x7f75000b2fd0 2023-11-04T05:28:51.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.100+0000 7f75059bb700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f74e406af00 0x7f74e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.101+0000 7f75059bb700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f74e406af00 0x7f74e406d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74f0000f60 tx=0x7f74f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.104+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f74fc055d60 con 0x7f75000b2fd0 2023-11-04T05:28:51.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.300+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 --> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75000b5c10 con 0x7f74e406af00 2023-11-04T05:28:51.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.311+0000 7f74f67fc700 1 -- 172.21.15.3:0/1089526332 <== mgr.4099 v2:172.21.15.188:6832/3102759174 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345258 (secure 0 0 0) 0x7f75000b5c10 con 0x7f74e406af00 2023-11-04T05:28:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:51.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f74e406af00 msgr2=0x7f74e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f74e406af00 0x7f74e406d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74f0000f60 tx=0x7f74f0009040 comp rx=0 tx=0).stop 2023-11-04T05:28:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 msgr2=0x7f75000b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 0x7f75000b3450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00ba10 tx=0x7f74fc0047c0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 shutdown_connections 2023-11-04T05:28:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f74e406af00 0x7f74e406d3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec004850 0x7f7500158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75000c9a00 0x7f75000c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 --2- 172.21.15.3:0/1089526332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75000b2fd0 0x7f75000b3450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 >> 172.21.15.3:0/1089526332 conn(0x7f750006df90 msgr2=0x7f7500160750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:51.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 shutdown_connections 2023-11-04T05:28:51.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.314+0000 7f7507c1f700 1 -- 172.21.15.3:0/1089526332 wait complete. 2023-11-04T05:28:51.322 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-11-04T05:28:51.328 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-11-04T05:28:49.757310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2045232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003676"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835820+0000","last_change":"2023-11-04T05:28:21.455216+0000","last_active":"2023-11-04T05:28:25.835820+0000","last_peered":"2023-11-04T05:28:25.835820+0000","last_clean":"2023-11-04T05:28:25.835820+0000","last_became_active":"2023-11-04T05:28:21.453964+0000","last_became_peered":"2023-11-04T05:28:21.453964+0000","last_unstale":"2023-11-04T05:28:25.835820+0000","last_undegraded":"2023-11-04T05:28:25.835820+0000","last_fullsized":"2023-11-04T05:28:25.835820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858340+0000","last_change":"2023-11-04T05:28:21.458543+0000","last_active":"2023-11-04T05:28:25.858340+0000","last_peered":"2023-11-04T05:28:25.858340+0000","last_clean":"2023-11-04T05:28:25.858340+0000","last_became_active":"2023-11-04T05:28:21.458174+0000","last_became_peered":"2023-11-04T05:28:21.458174+0000","last_unstale":"2023-11-04T05:28:25.858340+0000","last_undegraded":"2023-11-04T05:28:25.858340+0000","last_fullsized":"2023-11-04T05:28:25.858340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842597+0000","last_change":"2023-11-04T05:28:21.457190+0000","last_active":"2023-11-04T05:28:25.842597+0000","last_peered":"2023-11-04T05:28:25.842597+0000","last_clean":"2023-11-04T05:28:25.842597+0000","last_became_active":"2023-11-04T05:28:21.456901+0000","last_became_peered":"2023-11-04T05:28:21.456901+0000","last_unstale":"2023-11-04T05:28:25.842597+0000","last_undegraded":"2023-11-04T05:28:25.842597+0000","last_fullsized":"2023-11-04T05:28:25.842597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:00.273535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840758+0000","last_change":"2023-11-04T05:28:21.464308+0000","last_active":"2023-11-04T05:28:25.840758+0000","last_peered":"2023-11-04T05:28:25.840758+0000","last_clean":"2023-11-04T05:28:25.840758+0000","last_became_active":"2023-11-04T05:28:21.464035+0000","last_became_peered":"2023-11-04T05:28:21.464035+0000","last_unstale":"2023-11-04T05:28:25.840758+0000","last_undegraded":"2023-11-04T05:28:25.840758+0000","last_fullsized":"2023-11-04T05:28:25.840758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:21.443272+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:21.442885+0000","last_became_peered":"2023-11-04T05:28:21.442885+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839412+0000","last_change":"2023-11-04T05:28:21.461562+0000","last_active":"2023-11-04T05:28:25.839412+0000","last_peered":"2023-11-04T05:28:25.839412+0000","last_clean":"2023-11-04T05:28:25.839412+0000","last_became_active":"2023-11-04T05:28:21.461276+0000","last_became_peered":"2023-11-04T05:28:21.461276+0000","last_unstale":"2023-11-04T05:28:25.839412+0000","last_undegraded":"2023-11-04T05:28:25.839412+0000","last_fullsized":"2023-11-04T05:28:25.839412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834303+0000","last_change":"2023-11-04T05:28:21.456621+0000","last_active":"2023-11-04T05:28:25.834303+0000","last_peered":"2023-11-04T05:28:25.834303+0000","last_clean":"2023-11-04T05:28:25.834303+0000","last_became_active":"2023-11-04T05:28:21.456356+0000","last_became_peered":"2023-11-04T05:28:21.456356+0000","last_unstale":"2023-11-04T05:28:25.834303+0000","last_undegraded":"2023-11-04T05:28:25.834303+0000","last_fullsized":"2023-11-04T05:28:25.834303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841746+0000","last_change":"2023-11-04T05:28:21.463624+0000","last_active":"2023-11-04T05:28:25.841746+0000","last_peered":"2023-11-04T05:28:25.841746+0000","last_clean":"2023-11-04T05:28:25.841746+0000","last_became_active":"2023-11-04T05:28:21.463336+0000","last_became_peered":"2023-11-04T05:28:21.463336+0000","last_unstale":"2023-11-04T05:28:25.841746+0000","last_undegraded":"2023-11-04T05:28:25.841746+0000","last_fullsized":"2023-11-04T05:28:25.841746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835730+0000","last_change":"2023-11-04T05:28:21.466992+0000","last_active":"2023-11-04T05:28:25.835730+0000","last_peered":"2023-11-04T05:28:25.835730+0000","last_clean":"2023-11-04T05:28:25.835730+0000","last_became_active":"2023-11-04T05:28:21.466638+0000","last_became_peered":"2023-11-04T05:28:21.466638+0000","last_unstale":"2023-11-04T05:28:25.835730+0000","last_undegraded":"2023-11-04T05:28:25.835730+0000","last_fullsized":"2023-11-04T05:28:25.835730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:01:55.625411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835857+0000","last_change":"2023-11-04T05:28:21.466265+0000","last_active":"2023-11-04T05:28:25.835857+0000","last_peered":"2023-11-04T05:28:25.835857+0000","last_clean":"2023-11-04T05:28:25.835857+0000","last_became_active":"2023-11-04T05:28:21.466080+0000","last_became_peered":"2023-11-04T05:28:21.466080+0000","last_unstale":"2023-11-04T05:28:25.835857+0000","last_undegraded":"2023-11-04T05:28:25.835857+0000","last_fullsized":"2023-11-04T05:28:25.835857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835259+0000","last_change":"2023-11-04T05:28:21.465839+0000","last_active":"2023-11-04T05:28:25.835259+0000","last_peered":"2023-11-04T05:28:25.835259+0000","last_clean":"2023-11-04T05:28:25.835259+0000","last_became_active":"2023-11-04T05:28:21.465571+0000","last_became_peered":"2023-11-04T05:28:21.465571+0000","last_unstale":"2023-11-04T05:28:25.835259+0000","last_undegraded":"2023-11-04T05:28:25.835259+0000","last_fullsized":"2023-11-04T05:28:25.835259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843187+0000","last_change":"2023-11-04T05:28:21.459690+0000","last_active":"2023-11-04T05:28:25.843187+0000","last_peered":"2023-11-04T05:28:25.843187+0000","last_clean":"2023-11-04T05:28:25.843187+0000","last_became_active":"2023-11-04T05:28:21.459406+0000","last_became_peered":"2023-11-04T05:28:21.459406+0000","last_unstale":"2023-11-04T05:28:25.843187+0000","last_undegraded":"2023-11-04T05:28:25.843187+0000","last_fullsized":"2023-11-04T05:28:25.843187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840245+0000","last_change":"2023-11-04T05:28:21.452009+0000","last_active":"2023-11-04T05:28:25.840245+0000","last_peered":"2023-11-04T05:28:25.840245+0000","last_clean":"2023-11-04T05:28:25.840245+0000","last_became_active":"2023-11-04T05:28:21.451701+0000","last_became_peered":"2023-11-04T05:28:21.451701+0000","last_unstale":"2023-11-04T05:28:25.840245+0000","last_undegraded":"2023-11-04T05:28:25.840245+0000","last_fullsized":"2023-11-04T05:28:25.840245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842267+0000","last_change":"2023-11-04T05:28:21.449054+0000","last_active":"2023-11-04T05:28:25.842267+0000","last_peered":"2023-11-04T05:28:25.842267+0000","last_clean":"2023-11-04T05:28:25.842267+0000","last_became_active":"2023-11-04T05:28:21.448593+0000","last_became_peered":"2023-11-04T05:28:21.448593+0000","last_unstale":"2023-11-04T05:28:25.842267+0000","last_undegraded":"2023-11-04T05:28:25.842267+0000","last_fullsized":"2023-11-04T05:28:25.842267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:16:04.673589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857682+0000","last_change":"2023-11-04T05:28:21.459859+0000","last_active":"2023-11-04T05:28:25.857682+0000","last_peered":"2023-11-04T05:28:25.857682+0000","last_clean":"2023-11-04T05:28:25.857682+0000","last_became_active":"2023-11-04T05:28:21.459569+0000","last_became_peered":"2023-11-04T05:28:21.459569+0000","last_unstale":"2023-11-04T05:28:25.857682+0000","last_undegraded":"2023-11-04T05:28:25.857682+0000","last_fullsized":"2023-11-04T05:28:25.857682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839549+0000","last_change":"2023-11-04T05:28:21.461640+0000","last_active":"2023-11-04T05:28:25.839549+0000","last_peered":"2023-11-04T05:28:25.839549+0000","last_clean":"2023-11-04T05:28:25.839549+0000","last_became_active":"2023-11-04T05:28:21.461239+0000","last_became_peered":"2023-11-04T05:28:21.461239+0000","last_unstale":"2023-11-04T05:28:25.839549+0000","last_undegraded":"2023-11-04T05:28:25.839549+0000","last_fullsized":"2023-11-04T05:28:25.839549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841199+0000","last_change":"2023-11-04T05:28:21.463999+0000","last_active":"2023-11-04T05:28:25.841199+0000","last_peered":"2023-11-04T05:28:25.841199+0000","last_clean":"2023-11-04T05:28:25.841199+0000","last_became_active":"2023-11-04T05:28:21.463743+0000","last_became_peered":"2023-11-04T05:28:21.463743+0000","last_unstale":"2023-11-04T05:28:25.841199+0000","last_undegraded":"2023-11-04T05:28:25.841199+0000","last_fullsized":"2023-11-04T05:28:25.841199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:53.811333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846908+0000","last_change":"2023-11-04T05:28:21.454396+0000","last_active":"2023-11-04T05:28:25.846908+0000","last_peered":"2023-11-04T05:28:25.846908+0000","last_clean":"2023-11-04T05:28:25.846908+0000","last_became_active":"2023-11-04T05:28:21.454074+0000","last_became_peered":"2023-11-04T05:28:21.454074+0000","last_unstale":"2023-11-04T05:28:25.846908+0000","last_undegraded":"2023-11-04T05:28:25.846908+0000","last_fullsized":"2023-11-04T05:28:25.846908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:23:43.407199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835087+0000","last_change":"2023-11-04T05:28:21.441379+0000","last_active":"2023-11-04T05:28:25.835087+0000","last_peered":"2023-11-04T05:28:25.835087+0000","last_clean":"2023-11-04T05:28:25.835087+0000","last_became_active":"2023-11-04T05:28:21.441074+0000","last_became_peered":"2023-11-04T05:28:21.441074+0000","last_unstale":"2023-11-04T05:28:25.835087+0000","last_undegraded":"2023-11-04T05:28:25.835087+0000","last_fullsized":"2023-11-04T05:28:25.835087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220068+0000","last_change":"2023-11-04T05:28:25.856956+0000","last_active":"2023-11-04T05:28:28.220068+0000","last_peered":"2023-11-04T05:28:28.220068+0000","last_clean":"2023-11-04T05:28:28.220068+0000","last_became_active":"2023-11-04T05:28:25.856458+0000","last_became_peered":"2023-11-04T05:28:25.856458+0000","last_unstale":"2023-11-04T05:28:28.220068+0000","last_undegraded":"2023-11-04T05:28:28.220068+0000","last_fullsized":"2023-11-04T05:28:28.220068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:01:17.491615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842941+0000","last_change":"2023-11-04T05:28:20.455408+0000","last_active":"2023-11-04T05:28:25.842941+0000","last_peered":"2023-11-04T05:28:25.842941+0000","last_clean":"2023-11-04T05:28:25.842941+0000","last_became_active":"2023-11-04T05:28:20.455106+0000","last_became_peered":"2023-11-04T05:28:20.455106+0000","last_unstale":"2023-11-04T05:28:25.842941+0000","last_undegraded":"2023-11-04T05:28:25.842941+0000","last_fullsized":"2023-11-04T05:28:25.842941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:09:15.898778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840543+0000","last_change":"2023-11-04T05:28:21.452315+0000","last_active":"2023-11-04T05:28:25.840543+0000","last_peered":"2023-11-04T05:28:25.840543+0000","last_clean":"2023-11-04T05:28:25.840543+0000","last_became_active":"2023-11-04T05:28:21.451728+0000","last_became_peered":"2023-11-04T05:28:21.451728+0000","last_unstale":"2023-11-04T05:28:25.840543+0000","last_undegraded":"2023-11-04T05:28:25.840543+0000","last_fullsized":"2023-11-04T05:28:25.840543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742885+0000","last_change":"2023-11-04T05:28:25.861875+0000","last_active":"2023-11-04T05:28:28.742885+0000","last_peered":"2023-11-04T05:28:28.742885+0000","last_clean":"2023-11-04T05:28:28.742885+0000","last_became_active":"2023-11-04T05:28:25.861458+0000","last_became_peered":"2023-11-04T05:28:25.861458+0000","last_unstale":"2023-11-04T05:28:28.742885+0000","last_undegraded":"2023-11-04T05:28:28.742885+0000","last_fullsized":"2023-11-04T05:28:28.742885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.153647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840648+0000","last_change":"2023-11-04T05:28:20.456506+0000","last_active":"2023-11-04T05:28:25.840648+0000","last_peered":"2023-11-04T05:28:25.840648+0000","last_clean":"2023-11-04T05:28:25.840648+0000","last_became_active":"2023-11-04T05:28:20.456086+0000","last_became_peered":"2023-11-04T05:28:20.456086+0000","last_unstale":"2023-11-04T05:28:25.840648+0000","last_undegraded":"2023-11-04T05:28:25.840648+0000","last_fullsized":"2023-11-04T05:28:25.840648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842845+0000","last_change":"2023-11-04T05:28:21.464433+0000","last_active":"2023-11-04T05:28:25.842845+0000","last_peered":"2023-11-04T05:28:25.842845+0000","last_clean":"2023-11-04T05:28:25.842845+0000","last_became_active":"2023-11-04T05:28:21.464133+0000","last_became_peered":"2023-11-04T05:28:21.464133+0000","last_unstale":"2023-11-04T05:28:25.842845+0000","last_undegraded":"2023-11-04T05:28:25.842845+0000","last_fullsized":"2023-11-04T05:28:25.842845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:26:07.829857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743096+0000","last_change":"2023-11-04T05:28:25.858693+0000","last_active":"2023-11-04T05:28:28.743096+0000","last_peered":"2023-11-04T05:28:28.743096+0000","last_clean":"2023-11-04T05:28:28.743096+0000","last_became_active":"2023-11-04T05:28:25.858151+0000","last_became_peered":"2023-11-04T05:28:25.858151+0000","last_unstale":"2023-11-04T05:28:28.743096+0000","last_undegraded":"2023-11-04T05:28:28.743096+0000","last_fullsized":"2023-11-04T05:28:28.743096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:33:57.728742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834712+0000","last_change":"2023-11-04T05:28:20.448994+0000","last_active":"2023-11-04T05:28:25.834712+0000","last_peered":"2023-11-04T05:28:25.834712+0000","last_clean":"2023-11-04T05:28:25.834712+0000","last_became_active":"2023-11-04T05:28:20.448722+0000","last_became_peered":"2023-11-04T05:28:20.448722+0000","last_unstale":"2023-11-04T05:28:25.834712+0000","last_undegraded":"2023-11-04T05:28:25.834712+0000","last_fullsized":"2023-11-04T05:28:25.834712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839002+0000","last_change":"2023-11-04T05:28:21.464488+0000","last_active":"2023-11-04T05:28:25.839002+0000","last_peered":"2023-11-04T05:28:25.839002+0000","last_clean":"2023-11-04T05:28:25.839002+0000","last_became_active":"2023-11-04T05:28:21.462679+0000","last_became_peered":"2023-11-04T05:28:21.462679+0000","last_unstale":"2023-11-04T05:28:25.839002+0000","last_undegraded":"2023-11-04T05:28:25.839002+0000","last_fullsized":"2023-11-04T05:28:25.839002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743030+0000","last_change":"2023-11-04T05:28:25.854516+0000","last_active":"2023-11-04T05:28:28.743030+0000","last_peered":"2023-11-04T05:28:28.743030+0000","last_clean":"2023-11-04T05:28:28.743030+0000","last_became_active":"2023-11-04T05:28:25.854162+0000","last_became_peered":"2023-11-04T05:28:25.854162+0000","last_unstale":"2023-11-04T05:28:28.743030+0000","last_undegraded":"2023-11-04T05:28:28.743030+0000","last_fullsized":"2023-11-04T05:28:28.743030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:06.645428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834998+0000","last_change":"2023-11-04T05:28:20.452770+0000","last_active":"2023-11-04T05:28:25.834998+0000","last_peered":"2023-11-04T05:28:25.834998+0000","last_clean":"2023-11-04T05:28:25.834998+0000","last_became_active":"2023-11-04T05:28:20.452404+0000","last_became_peered":"2023-11-04T05:28:20.452404+0000","last_unstale":"2023-11-04T05:28:25.834998+0000","last_undegraded":"2023-11-04T05:28:25.834998+0000","last_fullsized":"2023-11-04T05:28:25.834998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842144+0000","last_change":"2023-11-04T05:28:21.463294+0000","last_active":"2023-11-04T05:28:25.842144+0000","last_peered":"2023-11-04T05:28:25.842144+0000","last_clean":"2023-11-04T05:28:25.842144+0000","last_became_active":"2023-11-04T05:28:21.463028+0000","last_became_peered":"2023-11-04T05:28:21.463028+0000","last_unstale":"2023-11-04T05:28:25.842144+0000","last_undegraded":"2023-11-04T05:28:25.842144+0000","last_fullsized":"2023-11-04T05:28:25.842144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220306+0000","last_change":"2023-11-04T05:28:25.864434+0000","last_active":"2023-11-04T05:28:28.220306+0000","last_peered":"2023-11-04T05:28:28.220306+0000","last_clean":"2023-11-04T05:28:28.220306+0000","last_became_active":"2023-11-04T05:28:25.863978+0000","last_became_peered":"2023-11-04T05:28:25.863978+0000","last_unstale":"2023-11-04T05:28:28.220306+0000","last_undegraded":"2023-11-04T05:28:28.220306+0000","last_fullsized":"2023-11-04T05:28:28.220306+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:22:02.298285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840436+0000","last_change":"2023-11-04T05:28:20.451845+0000","last_active":"2023-11-04T05:28:25.840436+0000","last_peered":"2023-11-04T05:28:25.840436+0000","last_clean":"2023-11-04T05:28:25.840436+0000","last_became_active":"2023-11-04T05:28:20.451350+0000","last_became_peered":"2023-11-04T05:28:20.451350+0000","last_unstale":"2023-11-04T05:28:25.840436+0000","last_undegraded":"2023-11-04T05:28:25.840436+0000","last_fullsized":"2023-11-04T05:28:25.840436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.837664+0000","last_change":"2023-11-04T05:28:21.454758+0000","last_active":"2023-11-04T05:28:25.837664+0000","last_peered":"2023-11-04T05:28:25.837664+0000","last_clean":"2023-11-04T05:28:25.837664+0000","last_became_active":"2023-11-04T05:28:21.449696+0000","last_became_peered":"2023-11-04T05:28:21.449696+0000","last_unstale":"2023-11-04T05:28:25.837664+0000","last_undegraded":"2023-11-04T05:28:25.837664+0000","last_fullsized":"2023-11-04T05:28:25.837664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:13:54.163937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.177060+0000","last_change":"2023-11-04T05:28:25.869885+0000","last_active":"2023-11-04T05:28:28.177060+0000","last_peered":"2023-11-04T05:28:28.177060+0000","last_clean":"2023-11-04T05:28:28.177060+0000","last_became_active":"2023-11-04T05:28:25.869295+0000","last_became_peered":"2023-11-04T05:28:25.869295+0000","last_unstale":"2023-11-04T05:28:28.177060+0000","last_undegraded":"2023-11-04T05:28:28.177060+0000","last_fullsized":"2023-11-04T05:28:28.177060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:02:41.884046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834648+0000","last_change":"2023-11-04T05:28:20.445804+0000","last_active":"2023-11-04T05:28:25.834648+0000","last_peered":"2023-11-04T05:28:25.834648+0000","last_clean":"2023-11-04T05:28:25.834648+0000","last_became_active":"2023-11-04T05:28:20.445476+0000","last_became_peered":"2023-11-04T05:28:20.445476+0000","last_unstale":"2023-11-04T05:28:25.834648+0000","last_undegraded":"2023-11-04T05:28:25.834648+0000","last_fullsized":"2023-11-04T05:28:25.834648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:22:03.756669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842664+0000","last_change":"2023-11-04T05:28:21.460996+0000","last_active":"2023-11-04T05:28:25.842664+0000","last_peered":"2023-11-04T05:28:25.842664+0000","last_clean":"2023-11-04T05:28:25.842664+0000","last_became_active":"2023-11-04T05:28:21.460451+0000","last_became_peered":"2023-11-04T05:28:21.460451+0000","last_unstale":"2023-11-04T05:28:25.842664+0000","last_undegraded":"2023-11-04T05:28:25.842664+0000","last_fullsized":"2023-11-04T05:28:25.842664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:16:02.249647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838965+0000","last_change":"2023-11-04T05:28:25.863990+0000","last_active":"2023-11-04T05:28:26.838965+0000","last_peered":"2023-11-04T05:28:26.838965+0000","last_clean":"2023-11-04T05:28:26.838965+0000","last_became_active":"2023-11-04T05:28:25.863587+0000","last_became_peered":"2023-11-04T05:28:25.863587+0000","last_unstale":"2023-11-04T05:28:26.838965+0000","last_undegraded":"2023-11-04T05:28:26.838965+0000","last_fullsized":"2023-11-04T05:28:26.838965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841913+0000","last_change":"2023-11-04T05:28:20.448359+0000","last_active":"2023-11-04T05:28:25.841913+0000","last_peered":"2023-11-04T05:28:25.841913+0000","last_clean":"2023-11-04T05:28:25.841913+0000","last_became_active":"2023-11-04T05:28:20.448037+0000","last_became_peered":"2023-11-04T05:28:20.448037+0000","last_unstale":"2023-11-04T05:28:25.841913+0000","last_undegraded":"2023-11-04T05:28:25.841913+0000","last_fullsized":"2023-11-04T05:28:25.841913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:02:08.721929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839909+0000","last_change":"2023-11-04T05:28:21.459362+0000","last_active":"2023-11-04T05:28:25.839909+0000","last_peered":"2023-11-04T05:28:25.839909+0000","last_clean":"2023-11-04T05:28:25.839909+0000","last_became_active":"2023-11-04T05:28:21.458878+0000","last_became_peered":"2023-11-04T05:28:21.458878+0000","last_unstale":"2023-11-04T05:28:25.839909+0000","last_undegraded":"2023-11-04T05:28:25.839909+0000","last_fullsized":"2023-11-04T05:28:25.839909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919977+0000","last_change":"2023-11-04T05:28:25.865090+0000","last_active":"2023-11-04T05:28:27.919977+0000","last_peered":"2023-11-04T05:28:27.919977+0000","last_clean":"2023-11-04T05:28:27.919977+0000","last_became_active":"2023-11-04T05:28:25.864492+0000","last_became_peered":"2023-11-04T05:28:25.864492+0000","last_unstale":"2023-11-04T05:28:27.919977+0000","last_undegraded":"2023-11-04T05:28:27.919977+0000","last_fullsized":"2023-11-04T05:28:27.919977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:52:12.074596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839654+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.839654+0000","last_peered":"2023-11-04T05:28:25.839654+0000","last_clean":"2023-11-04T05:28:25.839654+0000","last_became_active":"2023-11-04T05:28:20.451322+0000","last_became_peered":"2023-11-04T05:28:20.451322+0000","last_unstale":"2023-11-04T05:28:25.839654+0000","last_undegraded":"2023-11-04T05:28:25.839654+0000","last_fullsized":"2023-11-04T05:28:25.839654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:16.770342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857938+0000","last_change":"2023-11-04T05:28:21.446085+0000","last_active":"2023-11-04T05:28:25.857938+0000","last_peered":"2023-11-04T05:28:25.857938+0000","last_clean":"2023-11-04T05:28:25.857938+0000","last_became_active":"2023-11-04T05:28:21.445831+0000","last_became_peered":"2023-11-04T05:28:21.445831+0000","last_unstale":"2023-11-04T05:28:25.857938+0000","last_undegraded":"2023-11-04T05:28:25.857938+0000","last_fullsized":"2023-11-04T05:28:25.857938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219926+0000","last_change":"2023-11-04T05:28:25.853036+0000","last_active":"2023-11-04T05:28:28.219926+0000","last_peered":"2023-11-04T05:28:28.219926+0000","last_clean":"2023-11-04T05:28:28.219926+0000","last_became_active":"2023-11-04T05:28:25.852217+0000","last_became_peered":"2023-11-04T05:28:25.852217+0000","last_unstale":"2023-11-04T05:28:28.219926+0000","last_undegraded":"2023-11-04T05:28:28.219926+0000","last_fullsized":"2023-11-04T05:28:28.219926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835077+0000","last_change":"2023-11-04T05:28:20.448588+0000","last_active":"2023-11-04T05:28:25.835077+0000","last_peered":"2023-11-04T05:28:25.835077+0000","last_clean":"2023-11-04T05:28:25.835077+0000","last_became_active":"2023-11-04T05:28:20.448330+0000","last_became_peered":"2023-11-04T05:28:20.448330+0000","last_unstale":"2023-11-04T05:28:25.835077+0000","last_undegraded":"2023-11-04T05:28:25.835077+0000","last_fullsized":"2023-11-04T05:28:25.835077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841018+0000","last_change":"2023-11-04T05:28:21.462557+0000","last_active":"2023-11-04T05:28:25.841018+0000","last_peered":"2023-11-04T05:28:25.841018+0000","last_clean":"2023-11-04T05:28:25.841018+0000","last_became_active":"2023-11-04T05:28:21.462247+0000","last_became_peered":"2023-11-04T05:28:21.462247+0000","last_unstale":"2023-11-04T05:28:25.841018+0000","last_undegraded":"2023-11-04T05:28:25.841018+0000","last_fullsized":"2023-11-04T05:28:25.841018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219973+0000","last_change":"2023-11-04T05:28:25.860094+0000","last_active":"2023-11-04T05:28:28.219973+0000","last_peered":"2023-11-04T05:28:28.219973+0000","last_clean":"2023-11-04T05:28:28.219973+0000","last_became_active":"2023-11-04T05:28:25.859164+0000","last_became_peered":"2023-11-04T05:28:25.859164+0000","last_unstale":"2023-11-04T05:28:28.219973+0000","last_undegraded":"2023-11-04T05:28:28.219973+0000","last_fullsized":"2023-11-04T05:28:28.219973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840932+0000","last_change":"2023-11-04T05:28:20.451529+0000","last_active":"2023-11-04T05:28:25.840932+0000","last_peered":"2023-11-04T05:28:25.840932+0000","last_clean":"2023-11-04T05:28:25.840932+0000","last_became_active":"2023-11-04T05:28:20.451236+0000","last_became_peered":"2023-11-04T05:28:20.451236+0000","last_unstale":"2023-11-04T05:28:25.840932+0000","last_undegraded":"2023-11-04T05:28:25.840932+0000","last_fullsized":"2023-11-04T05:28:25.840932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:30.058271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843021+0000","last_change":"2023-11-04T05:28:21.464721+0000","last_active":"2023-11-04T05:28:25.843021+0000","last_peered":"2023-11-04T05:28:25.843021+0000","last_clean":"2023-11-04T05:28:25.843021+0000","last_became_active":"2023-11-04T05:28:21.464403+0000","last_became_peered":"2023-11-04T05:28:21.464403+0000","last_unstale":"2023-11-04T05:28:25.843021+0000","last_undegraded":"2023-11-04T05:28:25.843021+0000","last_fullsized":"2023-11-04T05:28:25.843021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:40.757866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743335+0000","last_change":"2023-11-04T05:28:25.858588+0000","last_active":"2023-11-04T05:28:28.743335+0000","last_peered":"2023-11-04T05:28:28.743335+0000","last_clean":"2023-11-04T05:28:28.743335+0000","last_became_active":"2023-11-04T05:28:25.858202+0000","last_became_peered":"2023-11-04T05:28:25.858202+0000","last_unstale":"2023-11-04T05:28:28.743335+0000","last_undegraded":"2023-11-04T05:28:28.743335+0000","last_fullsized":"2023-11-04T05:28:28.743335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:36:16.890049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858629+0000","last_change":"2023-11-04T05:28:20.451067+0000","last_active":"2023-11-04T05:28:25.858629+0000","last_peered":"2023-11-04T05:28:25.858629+0000","last_clean":"2023-11-04T05:28:25.858629+0000","last_became_active":"2023-11-04T05:28:20.450732+0000","last_became_peered":"2023-11-04T05:28:20.450732+0000","last_unstale":"2023-11-04T05:28:25.858629+0000","last_undegraded":"2023-11-04T05:28:25.858629+0000","last_fullsized":"2023-11-04T05:28:25.858629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843093+0000","last_change":"2023-11-04T05:28:21.457579+0000","last_active":"2023-11-04T05:28:25.843093+0000","last_peered":"2023-11-04T05:28:25.843093+0000","last_clean":"2023-11-04T05:28:25.843093+0000","last_became_active":"2023-11-04T05:28:21.457314+0000","last_became_peered":"2023-11-04T05:28:21.457314+0000","last_unstale":"2023-11-04T05:28:25.843093+0000","last_undegraded":"2023-11-04T05:28:25.843093+0000","last_fullsized":"2023-11-04T05:28:25.843093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832422+0000","last_change":"2023-11-04T05:28:25.863159+0000","last_active":"2023-11-04T05:28:28.832422+0000","last_peered":"2023-11-04T05:28:28.832422+0000","last_clean":"2023-11-04T05:28:28.832422+0000","last_became_active":"2023-11-04T05:28:25.862645+0000","last_became_peered":"2023-11-04T05:28:25.862645+0000","last_unstale":"2023-11-04T05:28:28.832422+0000","last_undegraded":"2023-11-04T05:28:28.832422+0000","last_fullsized":"2023-11-04T05:28:28.832422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:41.867932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:31.743983+0000","last_change":"2023-11-04T05:28:20.449394+0000","last_active":"2023-11-04T05:28:31.743983+0000","last_peered":"2023-11-04T05:28:31.743983+0000","last_clean":"2023-11-04T05:28:31.743983+0000","last_became_active":"2023-11-04T05:28:20.449060+0000","last_became_peered":"2023-11-04T05:28:20.449060+0000","last_unstale":"2023-11-04T05:28:31.743983+0000","last_undegraded":"2023-11-04T05:28:31.743983+0000","last_fullsized":"2023-11-04T05:28:31.743983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835007+0000","last_change":"2023-11-04T05:28:21.459298+0000","last_active":"2023-11-04T05:28:25.835007+0000","last_peered":"2023-11-04T05:28:25.835007+0000","last_clean":"2023-11-04T05:28:25.835007+0000","last_became_active":"2023-11-04T05:28:21.459046+0000","last_became_peered":"2023-11-04T05:28:21.459046+0000","last_unstale":"2023-11-04T05:28:25.835007+0000","last_undegraded":"2023-11-04T05:28:25.835007+0000","last_fullsized":"2023-11-04T05:28:25.835007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836097+0000","last_change":"2023-11-04T05:28:25.853324+0000","last_active":"2023-11-04T05:28:26.836097+0000","last_peered":"2023-11-04T05:28:26.836097+0000","last_clean":"2023-11-04T05:28:26.836097+0000","last_became_active":"2023-11-04T05:28:25.852913+0000","last_became_peered":"2023-11-04T05:28:25.852913+0000","last_unstale":"2023-11-04T05:28:26.836097+0000","last_undegraded":"2023-11-04T05:28:26.836097+0000","last_fullsized":"2023-11-04T05:28:26.836097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:27:07.538051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842933+0000","last_change":"2023-11-04T05:28:20.451652+0000","last_active":"2023-11-04T05:28:25.842933+0000","last_peered":"2023-11-04T05:28:25.842933+0000","last_clean":"2023-11-04T05:28:25.842933+0000","last_became_active":"2023-11-04T05:28:20.451415+0000","last_became_peered":"2023-11-04T05:28:20.451415+0000","last_unstale":"2023-11-04T05:28:25.842933+0000","last_undegraded":"2023-11-04T05:28:25.842933+0000","last_fullsized":"2023-11-04T05:28:25.842933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:05:45.132028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842574+0000","last_change":"2023-11-04T05:28:21.454210+0000","last_active":"2023-11-04T05:28:25.842574+0000","last_peered":"2023-11-04T05:28:25.842574+0000","last_clean":"2023-11-04T05:28:25.842574+0000","last_became_active":"2023-11-04T05:28:21.453894+0000","last_became_peered":"2023-11-04T05:28:21.453894+0000","last_unstale":"2023-11-04T05:28:25.842574+0000","last_undegraded":"2023-11-04T05:28:25.842574+0000","last_fullsized":"2023-11-04T05:28:25.842574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:28:59.983010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220347+0000","last_change":"2023-11-04T05:28:25.864286+0000","last_active":"2023-11-04T05:28:28.220347+0000","last_peered":"2023-11-04T05:28:28.220347+0000","last_clean":"2023-11-04T05:28:28.220347+0000","last_became_active":"2023-11-04T05:28:25.863831+0000","last_became_peered":"2023-11-04T05:28:25.863831+0000","last_unstale":"2023-11-04T05:28:28.220347+0000","last_undegraded":"2023-11-04T05:28:28.220347+0000","last_fullsized":"2023-11-04T05:28:28.220347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:40:57.121972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839219+0000","last_change":"2023-11-04T05:28:20.456235+0000","last_active":"2023-11-04T05:28:25.839219+0000","last_peered":"2023-11-04T05:28:25.839219+0000","last_clean":"2023-11-04T05:28:25.839219+0000","last_became_active":"2023-11-04T05:28:20.455992+0000","last_became_peered":"2023-11-04T05:28:20.455992+0000","last_unstale":"2023-11-04T05:28:25.839219+0000","last_undegraded":"2023-11-04T05:28:25.839219+0000","last_fullsized":"2023-11-04T05:28:25.839219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834802+0000","last_change":"2023-11-04T05:28:21.460351+0000","last_active":"2023-11-04T05:28:25.834802+0000","last_peered":"2023-11-04T05:28:25.834802+0000","last_clean":"2023-11-04T05:28:25.834802+0000","last_became_active":"2023-11-04T05:28:21.459737+0000","last_became_peered":"2023-11-04T05:28:21.459737+0000","last_unstale":"2023-11-04T05:28:25.834802+0000","last_undegraded":"2023-11-04T05:28:25.834802+0000","last_fullsized":"2023-11-04T05:28:25.834802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:09.082834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.220525+0000","last_change":"2023-11-04T05:28:25.868351+0000","last_active":"2023-11-04T05:28:28.220525+0000","last_peered":"2023-11-04T05:28:28.220525+0000","last_clean":"2023-11-04T05:28:28.220525+0000","last_became_active":"2023-11-04T05:28:25.867909+0000","last_became_peered":"2023-11-04T05:28:25.867909+0000","last_unstale":"2023-11-04T05:28:28.220525+0000","last_undegraded":"2023-11-04T05:28:28.220525+0000","last_fullsized":"2023-11-04T05:28:28.220525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:08:06.813956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833857+0000","last_change":"2023-11-04T05:28:20.447033+0000","last_active":"2023-11-04T05:28:25.833857+0000","last_peered":"2023-11-04T05:28:25.833857+0000","last_clean":"2023-11-04T05:28:25.833857+0000","last_became_active":"2023-11-04T05:28:20.446767+0000","last_became_peered":"2023-11-04T05:28:20.446767+0000","last_unstale":"2023-11-04T05:28:25.833857+0000","last_undegraded":"2023-11-04T05:28:25.833857+0000","last_fullsized":"2023-11-04T05:28:25.833857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836084+0000","last_change":"2023-11-04T05:28:21.457261+0000","last_active":"2023-11-04T05:28:25.836084+0000","last_peered":"2023-11-04T05:28:25.836084+0000","last_clean":"2023-11-04T05:28:25.836084+0000","last_became_active":"2023-11-04T05:28:21.456982+0000","last_became_peered":"2023-11-04T05:28:21.456982+0000","last_unstale":"2023-11-04T05:28:25.836084+0000","last_undegraded":"2023-11-04T05:28:25.836084+0000","last_fullsized":"2023-11-04T05:28:25.836084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919838+0000","last_change":"2023-11-04T05:28:25.854585+0000","last_active":"2023-11-04T05:28:27.919838+0000","last_peered":"2023-11-04T05:28:27.919838+0000","last_clean":"2023-11-04T05:28:27.919838+0000","last_became_active":"2023-11-04T05:28:25.854151+0000","last_became_peered":"2023-11-04T05:28:25.854151+0000","last_unstale":"2023-11-04T05:28:27.919838+0000","last_undegraded":"2023-11-04T05:28:27.919838+0000","last_fullsized":"2023-11-04T05:28:27.919838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:05:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.844868+0000","last_change":"2023-11-04T05:28:20.448118+0000","last_active":"2023-11-04T05:28:25.844868+0000","last_peered":"2023-11-04T05:28:25.844868+0000","last_clean":"2023-11-04T05:28:25.844868+0000","last_became_active":"2023-11-04T05:28:20.447780+0000","last_became_peered":"2023-11-04T05:28:20.447780+0000","last_unstale":"2023-11-04T05:28:25.844868+0000","last_undegraded":"2023-11-04T05:28:25.844868+0000","last_fullsized":"2023-11-04T05:28:25.844868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:45:52.143624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834466+0000","last_change":"2023-11-04T05:28:21.458858+0000","last_active":"2023-11-04T05:28:25.834466+0000","last_peered":"2023-11-04T05:28:25.834466+0000","last_clean":"2023-11-04T05:28:25.834466+0000","last_became_active":"2023-11-04T05:28:21.458577+0000","last_became_peered":"2023-11-04T05:28:21.458577+0000","last_unstale":"2023-11-04T05:28:25.834466+0000","last_undegraded":"2023-11-04T05:28:25.834466+0000","last_fullsized":"2023-11-04T05:28:25.834466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742613+0000","last_change":"2023-11-04T05:28:25.865986+0000","last_active":"2023-11-04T05:28:28.742613+0000","last_peered":"2023-11-04T05:28:28.742613+0000","last_clean":"2023-11-04T05:28:28.742613+0000","last_became_active":"2023-11-04T05:28:25.865476+0000","last_became_peered":"2023-11-04T05:28:25.865476+0000","last_unstale":"2023-11-04T05:28:28.742613+0000","last_undegraded":"2023-11-04T05:28:28.742613+0000","last_fullsized":"2023-11-04T05:28:28.742613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:23.545163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842973+0000","last_change":"2023-11-04T05:28:20.451610+0000","last_active":"2023-11-04T05:28:25.842973+0000","last_peered":"2023-11-04T05:28:25.842973+0000","last_clean":"2023-11-04T05:28:25.842973+0000","last_became_active":"2023-11-04T05:28:20.451353+0000","last_became_peered":"2023-11-04T05:28:20.451353+0000","last_unstale":"2023-11-04T05:28:25.842973+0000","last_undegraded":"2023-11-04T05:28:25.842973+0000","last_fullsized":"2023-11-04T05:28:25.842973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:11:52.224724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835325+0000","last_change":"2023-11-04T05:28:21.460107+0000","last_active":"2023-11-04T05:28:25.835325+0000","last_peered":"2023-11-04T05:28:25.835325+0000","last_clean":"2023-11-04T05:28:25.835325+0000","last_became_active":"2023-11-04T05:28:21.458229+0000","last_became_peered":"2023-11-04T05:28:21.458229+0000","last_unstale":"2023-11-04T05:28:25.835325+0000","last_undegraded":"2023-11-04T05:28:25.835325+0000","last_fullsized":"2023-11-04T05:28:25.835325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832264+0000","last_change":"2023-11-04T05:28:25.863802+0000","last_active":"2023-11-04T05:28:28.832264+0000","last_peered":"2023-11-04T05:28:28.832264+0000","last_clean":"2023-11-04T05:28:28.832264+0000","last_became_active":"2023-11-04T05:28:25.863268+0000","last_became_peered":"2023-11-04T05:28:25.863268+0000","last_unstale":"2023-11-04T05:28:28.832264+0000","last_undegraded":"2023-11-04T05:28:28.832264+0000","last_fullsized":"2023-11-04T05:28:28.832264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:30.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862531+0000","last_change":"2023-11-04T05:28:20.453776+0000","last_active":"2023-11-04T05:28:25.862531+0000","last_peered":"2023-11-04T05:28:25.862531+0000","last_clean":"2023-11-04T05:28:25.862531+0000","last_became_active":"2023-11-04T05:28:20.453564+0000","last_became_peered":"2023-11-04T05:28:20.453564+0000","last_unstale":"2023-11-04T05:28:25.862531+0000","last_undegraded":"2023-11-04T05:28:25.862531+0000","last_fullsized":"2023-11-04T05:28:25.862531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842239+0000","last_change":"2023-11-04T05:28:21.463249+0000","last_active":"2023-11-04T05:28:25.842239+0000","last_peered":"2023-11-04T05:28:25.842239+0000","last_clean":"2023-11-04T05:28:25.842239+0000","last_became_active":"2023-11-04T05:28:21.462964+0000","last_became_peered":"2023-11-04T05:28:21.462964+0000","last_unstale":"2023-11-04T05:28:25.842239+0000","last_undegraded":"2023-11-04T05:28:25.842239+0000","last_fullsized":"2023-11-04T05:28:25.842239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:54:25.969114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832100+0000","last_change":"2023-11-04T05:28:25.866264+0000","last_active":"2023-11-04T05:28:28.832100+0000","last_peered":"2023-11-04T05:28:28.832100+0000","last_clean":"2023-11-04T05:28:28.832100+0000","last_became_active":"2023-11-04T05:28:25.865946+0000","last_became_peered":"2023-11-04T05:28:25.865946+0000","last_unstale":"2023-11-04T05:28:28.832100+0000","last_undegraded":"2023-11-04T05:28:28.832100+0000","last_fullsized":"2023-11-04T05:28:28.832100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:30.307509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839118+0000","last_change":"2023-11-04T05:28:20.449195+0000","last_active":"2023-11-04T05:28:25.839118+0000","last_peered":"2023-11-04T05:28:25.839118+0000","last_clean":"2023-11-04T05:28:25.839118+0000","last_became_active":"2023-11-04T05:28:20.448862+0000","last_became_peered":"2023-11-04T05:28:20.448862+0000","last_unstale":"2023-11-04T05:28:25.839118+0000","last_undegraded":"2023-11-04T05:28:25.839118+0000","last_fullsized":"2023-11-04T05:28:25.839118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:32.638827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843378+0000","last_change":"2023-11-04T05:28:21.464603+0000","last_active":"2023-11-04T05:28:25.843378+0000","last_peered":"2023-11-04T05:28:25.843378+0000","last_clean":"2023-11-04T05:28:25.843378+0000","last_became_active":"2023-11-04T05:28:21.464337+0000","last_became_peered":"2023-11-04T05:28:21.464337+0000","last_unstale":"2023-11-04T05:28:25.843378+0000","last_undegraded":"2023-11-04T05:28:25.843378+0000","last_fullsized":"2023-11-04T05:28:25.843378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:44.443386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832325+0000","last_change":"2023-11-04T05:28:25.864109+0000","last_active":"2023-11-04T05:28:28.832325+0000","last_peered":"2023-11-04T05:28:28.832325+0000","last_clean":"2023-11-04T05:28:28.832325+0000","last_became_active":"2023-11-04T05:28:25.863293+0000","last_became_peered":"2023-11-04T05:28:25.863293+0000","last_unstale":"2023-11-04T05:28:28.832325+0000","last_undegraded":"2023-11-04T05:28:28.832325+0000","last_fullsized":"2023-11-04T05:28:28.832325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:35:13.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857963+0000","last_change":"2023-11-04T05:28:21.448655+0000","last_active":"2023-11-04T05:28:25.857963+0000","last_peered":"2023-11-04T05:28:25.857963+0000","last_clean":"2023-11-04T05:28:25.857963+0000","last_became_active":"2023-11-04T05:28:21.448355+0000","last_became_peered":"2023-11-04T05:28:21.448355+0000","last_unstale":"2023-11-04T05:28:25.857963+0000","last_undegraded":"2023-11-04T05:28:25.857963+0000","last_fullsized":"2023-11-04T05:28:25.857963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845533+0000","last_change":"2023-11-04T05:28:20.453706+0000","last_active":"2023-11-04T05:28:25.845533+0000","last_peered":"2023-11-04T05:28:25.845533+0000","last_clean":"2023-11-04T05:28:25.845533+0000","last_became_active":"2023-11-04T05:28:20.453428+0000","last_became_peered":"2023-11-04T05:28:20.453428+0000","last_unstale":"2023-11-04T05:28:25.845533+0000","last_undegraded":"2023-11-04T05:28:25.845533+0000","last_fullsized":"2023-11-04T05:28:25.845533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:24:43.438258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838700+0000","last_change":"2023-11-04T05:28:25.852681+0000","last_active":"2023-11-04T05:28:26.838700+0000","last_peered":"2023-11-04T05:28:26.838700+0000","last_clean":"2023-11-04T05:28:26.838700+0000","last_became_active":"2023-11-04T05:28:25.852276+0000","last_became_peered":"2023-11-04T05:28:25.852276+0000","last_unstale":"2023-11-04T05:28:26.838700+0000","last_undegraded":"2023-11-04T05:28:26.838700+0000","last_fullsized":"2023-11-04T05:28:26.838700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:28.005917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840730+0000","last_change":"2023-11-04T05:28:21.466480+0000","last_active":"2023-11-04T05:28:25.840730+0000","last_peered":"2023-11-04T05:28:25.840730+0000","last_clean":"2023-11-04T05:28:25.840730+0000","last_became_active":"2023-11-04T05:28:21.466065+0000","last_became_peered":"2023-11-04T05:28:21.466065+0000","last_unstale":"2023-11-04T05:28:25.840730+0000","last_undegraded":"2023-11-04T05:28:25.840730+0000","last_fullsized":"2023-11-04T05:28:25.840730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:31.063399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.838807+0000","last_change":"2023-11-04T05:28:20.451336+0000","last_active":"2023-11-04T05:28:25.838807+0000","last_peered":"2023-11-04T05:28:25.838807+0000","last_clean":"2023-11-04T05:28:25.838807+0000","last_became_active":"2023-11-04T05:28:20.451000+0000","last_became_peered":"2023-11-04T05:28:20.451000+0000","last_unstale":"2023-11-04T05:28:25.838807+0000","last_undegraded":"2023-11-04T05:28:25.838807+0000","last_fullsized":"2023-11-04T05:28:25.838807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:46.798012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834100+0000","last_change":"2023-11-04T05:28:11.829985+0000","last_active":"2023-11-04T05:28:25.834100+0000","last_peered":"2023-11-04T05:28:25.834100+0000","last_clean":"2023-11-04T05:28:25.834100+0000","last_became_active":"2023-11-04T05:28:11.829705+0000","last_became_peered":"2023-11-04T05:28:11.829705+0000","last_unstale":"2023-11-04T05:28:25.834100+0000","last_undegraded":"2023-11-04T05:28:25.834100+0000","last_fullsized":"2023-11-04T05:28:25.834100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_clean_scrub_stamp":"2023-11-04T05:28:10.378947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:46:15.100605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919902+0000","last_change":"2023-11-04T05:28:25.865855+0000","last_active":"2023-11-04T05:28:27.919902+0000","last_peered":"2023-11-04T05:28:27.919902+0000","last_clean":"2023-11-04T05:28:27.919902+0000","last_became_active":"2023-11-04T05:28:25.865538+0000","last_became_peered":"2023-11-04T05:28:25.865538+0000","last_unstale":"2023-11-04T05:28:27.919902+0000","last_undegraded":"2023-11-04T05:28:27.919902+0000","last_fullsized":"2023-11-04T05:28:27.919902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:24:58.810725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842382+0000","last_change":"2023-11-04T05:28:21.461543+0000","last_active":"2023-11-04T05:28:25.842382+0000","last_peered":"2023-11-04T05:28:25.842382+0000","last_clean":"2023-11-04T05:28:25.842382+0000","last_became_active":"2023-11-04T05:28:21.461195+0000","last_became_peered":"2023-11-04T05:28:21.461195+0000","last_unstale":"2023-11-04T05:28:25.842382+0000","last_undegraded":"2023-11-04T05:28:25.842382+0000","last_fullsized":"2023-11-04T05:28:25.842382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:28:07.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834194+0000","last_change":"2023-11-04T05:28:20.450213+0000","last_active":"2023-11-04T05:28:25.834194+0000","last_peered":"2023-11-04T05:28:25.834194+0000","last_clean":"2023-11-04T05:28:25.834194+0000","last_became_active":"2023-11-04T05:28:20.449957+0000","last_became_peered":"2023-11-04T05:28:20.449957+0000","last_unstale":"2023-11-04T05:28:25.834194+0000","last_undegraded":"2023-11-04T05:28:25.834194+0000","last_fullsized":"2023-11-04T05:28:25.834194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:30:12.799059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919653+0000","last_change":"2023-11-04T05:28:25.861935+0000","last_active":"2023-11-04T05:28:27.919653+0000","last_peered":"2023-11-04T05:28:27.919653+0000","last_clean":"2023-11-04T05:28:27.919653+0000","last_became_active":"2023-11-04T05:28:25.861471+0000","last_became_peered":"2023-11-04T05:28:25.861471+0000","last_unstale":"2023-11-04T05:28:27.919653+0000","last_undegraded":"2023-11-04T05:28:27.919653+0000","last_fullsized":"2023-11-04T05:28:27.919653+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:53:47.409110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834605+0000","last_change":"2023-11-04T05:28:20.455821+0000","last_active":"2023-11-04T05:28:25.834605+0000","last_peered":"2023-11-04T05:28:25.834605+0000","last_clean":"2023-11-04T05:28:25.834605+0000","last_became_active":"2023-11-04T05:28:20.455538+0000","last_became_peered":"2023-11-04T05:28:20.455538+0000","last_unstale":"2023-11-04T05:28:25.834605+0000","last_undegraded":"2023-11-04T05:28:25.834605+0000","last_fullsized":"2023-11-04T05:28:25.834605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:24.784210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.862653+0000","last_change":"2023-11-04T05:28:21.446022+0000","last_active":"2023-11-04T05:28:25.862653+0000","last_peered":"2023-11-04T05:28:25.862653+0000","last_clean":"2023-11-04T05:28:25.862653+0000","last_became_active":"2023-11-04T05:28:21.445727+0000","last_became_peered":"2023-11-04T05:28:21.445727+0000","last_unstale":"2023-11-04T05:28:25.862653+0000","last_undegraded":"2023-11-04T05:28:25.862653+0000","last_fullsized":"2023-11-04T05:28:25.862653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.219703+0000","last_change":"2023-11-04T05:28:25.860350+0000","last_active":"2023-11-04T05:28:28.219703+0000","last_peered":"2023-11-04T05:28:28.219703+0000","last_clean":"2023-11-04T05:28:28.219703+0000","last_became_active":"2023-11-04T05:28:25.859724+0000","last_became_peered":"2023-11-04T05:28:25.859724+0000","last_unstale":"2023-11-04T05:28:28.219703+0000","last_undegraded":"2023-11-04T05:28:28.219703+0000","last_fullsized":"2023-11-04T05:28:28.219703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:33.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.356335+0000","last_change":"2023-11-04T05:28:20.449071+0000","last_active":"2023-11-04T05:28:26.356335+0000","last_peered":"2023-11-04T05:28:26.356335+0000","last_clean":"2023-11-04T05:28:26.356335+0000","last_became_active":"2023-11-04T05:28:20.448778+0000","last_became_peered":"2023-11-04T05:28:20.448778+0000","last_unstale":"2023-11-04T05:28:26.356335+0000","last_undegraded":"2023-11-04T05:28:26.356335+0000","last_fullsized":"2023-11-04T05:28:26.356335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:35:08.740372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840260+0000","last_change":"2023-11-04T05:28:21.444332+0000","last_active":"2023-11-04T05:28:25.840260+0000","last_peered":"2023-11-04T05:28:25.840260+0000","last_clean":"2023-11-04T05:28:25.840260+0000","last_became_active":"2023-11-04T05:28:21.444027+0000","last_became_peered":"2023-11-04T05:28:21.444027+0000","last_unstale":"2023-11-04T05:28:25.840260+0000","last_undegraded":"2023-11-04T05:28:25.840260+0000","last_fullsized":"2023-11-04T05:28:25.840260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.742687+0000","last_change":"2023-11-04T05:28:25.846982+0000","last_active":"2023-11-04T05:28:28.742687+0000","last_peered":"2023-11-04T05:28:28.742687+0000","last_clean":"2023-11-04T05:28:28.742687+0000","last_became_active":"2023-11-04T05:28:25.846562+0000","last_became_peered":"2023-11-04T05:28:25.846562+0000","last_unstale":"2023-11-04T05:28:28.742687+0000","last_undegraded":"2023-11-04T05:28:28.742687+0000","last_fullsized":"2023-11-04T05:28:28.742687+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:52.840764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834969+0000","last_change":"2023-11-04T05:28:20.447063+0000","last_active":"2023-11-04T05:28:25.834969+0000","last_peered":"2023-11-04T05:28:25.834969+0000","last_clean":"2023-11-04T05:28:25.834969+0000","last_became_active":"2023-11-04T05:28:20.446833+0000","last_became_peered":"2023-11-04T05:28:20.446833+0000","last_unstale":"2023-11-04T05:28:25.834969+0000","last_undegraded":"2023-11-04T05:28:25.834969+0000","last_fullsized":"2023-11-04T05:28:25.834969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:56:20.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842874+0000","last_change":"2023-11-04T05:28:21.456075+0000","last_active":"2023-11-04T05:28:25.842874+0000","last_peered":"2023-11-04T05:28:25.842874+0000","last_clean":"2023-11-04T05:28:25.842874+0000","last_became_active":"2023-11-04T05:28:21.455801+0000","last_became_peered":"2023-11-04T05:28:21.455801+0000","last_unstale":"2023-11-04T05:28:25.842874+0000","last_undegraded":"2023-11-04T05:28:25.842874+0000","last_fullsized":"2023-11-04T05:28:25.842874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:37:48.444983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743366+0000","last_change":"2023-11-04T05:28:25.864053+0000","last_active":"2023-11-04T05:28:28.743366+0000","last_peered":"2023-11-04T05:28:28.743366+0000","last_clean":"2023-11-04T05:28:28.743366+0000","last_became_active":"2023-11-04T05:28:25.863669+0000","last_became_peered":"2023-11-04T05:28:25.863669+0000","last_unstale":"2023-11-04T05:28:28.743366+0000","last_undegraded":"2023-11-04T05:28:28.743366+0000","last_fullsized":"2023-11-04T05:28:28.743366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:42:30.660108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.833998+0000","last_change":"2023-11-04T05:28:20.453827+0000","last_active":"2023-11-04T05:28:25.833998+0000","last_peered":"2023-11-04T05:28:25.833998+0000","last_clean":"2023-11-04T05:28:25.833998+0000","last_became_active":"2023-11-04T05:28:20.453576+0000","last_became_peered":"2023-11-04T05:28:20.453576+0000","last_unstale":"2023-11-04T05:28:25.833998+0000","last_undegraded":"2023-11-04T05:28:25.833998+0000","last_fullsized":"2023-11-04T05:28:25.833998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:40:55.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840329+0000","last_change":"2023-11-04T05:28:21.458086+0000","last_active":"2023-11-04T05:28:25.840329+0000","last_peered":"2023-11-04T05:28:25.840329+0000","last_clean":"2023-11-04T05:28:25.840329+0000","last_became_active":"2023-11-04T05:28:21.457739+0000","last_became_peered":"2023-11-04T05:28:21.457739+0000","last_unstale":"2023-11-04T05:28:25.840329+0000","last_undegraded":"2023-11-04T05:28:25.840329+0000","last_fullsized":"2023-11-04T05:28:25.840329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:45.088645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.832505+0000","last_change":"2023-11-04T05:28:25.864371+0000","last_active":"2023-11-04T05:28:28.832505+0000","last_peered":"2023-11-04T05:28:28.832505+0000","last_clean":"2023-11-04T05:28:28.832505+0000","last_became_active":"2023-11-04T05:28:25.863374+0000","last_became_peered":"2023-11-04T05:28:25.863374+0000","last_unstale":"2023-11-04T05:28:28.832505+0000","last_undegraded":"2023-11-04T05:28:28.832505+0000","last_fullsized":"2023-11-04T05:28:28.832505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:19:58.207114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.857793+0000","last_change":"2023-11-04T05:28:20.450039+0000","last_active":"2023-11-04T05:28:25.857793+0000","last_peered":"2023-11-04T05:28:25.857793+0000","last_clean":"2023-11-04T05:28:25.857793+0000","last_became_active":"2023-11-04T05:28:20.449761+0000","last_became_peered":"2023-11-04T05:28:20.449761+0000","last_unstale":"2023-11-04T05:28:25.857793+0000","last_undegraded":"2023-11-04T05:28:25.857793+0000","last_fullsized":"2023-11-04T05:28:25.857793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:53:59.043772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841634+0000","last_change":"2023-11-04T05:28:21.450153+0000","last_active":"2023-11-04T05:28:25.841634+0000","last_peered":"2023-11-04T05:28:25.841634+0000","last_clean":"2023-11-04T05:28:25.841634+0000","last_became_active":"2023-11-04T05:28:21.449878+0000","last_became_peered":"2023-11-04T05:28:21.449878+0000","last_unstale":"2023-11-04T05:28:25.841634+0000","last_undegraded":"2023-11-04T05:28:25.841634+0000","last_fullsized":"2023-11-04T05:28:25.841634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:45:33.643956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743302+0000","last_change":"2023-11-04T05:28:25.864439+0000","last_active":"2023-11-04T05:28:28.743302+0000","last_peered":"2023-11-04T05:28:28.743302+0000","last_clean":"2023-11-04T05:28:28.743302+0000","last_became_active":"2023-11-04T05:28:25.863909+0000","last_became_peered":"2023-11-04T05:28:25.863909+0000","last_unstale":"2023-11-04T05:28:28.743302+0000","last_undegraded":"2023-11-04T05:28:28.743302+0000","last_fullsized":"2023-11-04T05:28:28.743302+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:00:50.267120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858628+0000","last_change":"2023-11-04T05:28:20.450297+0000","last_active":"2023-11-04T05:28:25.858628+0000","last_peered":"2023-11-04T05:28:25.858628+0000","last_clean":"2023-11-04T05:28:25.858628+0000","last_became_active":"2023-11-04T05:28:20.450017+0000","last_became_peered":"2023-11-04T05:28:20.450017+0000","last_unstale":"2023-11-04T05:28:25.858628+0000","last_undegraded":"2023-11-04T05:28:25.858628+0000","last_fullsized":"2023-11-04T05:28:25.858628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:14:22.671964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840528+0000","last_change":"2023-11-04T05:28:21.457231+0000","last_active":"2023-11-04T05:28:25.840528+0000","last_peered":"2023-11-04T05:28:25.840528+0000","last_clean":"2023-11-04T05:28:25.840528+0000","last_became_active":"2023-11-04T05:28:21.456966+0000","last_became_peered":"2023-11-04T05:28:21.456966+0000","last_unstale":"2023-11-04T05:28:25.840528+0000","last_undegraded":"2023-11-04T05:28:25.840528+0000","last_fullsized":"2023-11-04T05:28:25.840528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:28.743383+0000","last_change":"2023-11-04T05:28:25.864991+0000","last_active":"2023-11-04T05:28:28.743383+0000","last_peered":"2023-11-04T05:28:28.743383+0000","last_clean":"2023-11-04T05:28:28.743383+0000","last_became_active":"2023-11-04T05:28:25.864122+0000","last_became_peered":"2023-11-04T05:28:25.864122+0000","last_unstale":"2023-11-04T05:28:28.743383+0000","last_undegraded":"2023-11-04T05:28:28.743383+0000","last_fullsized":"2023-11-04T05:28:28.743383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:28:31.685109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834888+0000","last_change":"2023-11-04T05:28:20.449101+0000","last_active":"2023-11-04T05:28:25.834888+0000","last_peered":"2023-11-04T05:28:25.834888+0000","last_clean":"2023-11-04T05:28:25.834888+0000","last_became_active":"2023-11-04T05:28:20.448809+0000","last_became_peered":"2023-11-04T05:28:20.448809+0000","last_unstale":"2023-11-04T05:28:25.834888+0000","last_undegraded":"2023-11-04T05:28:25.834888+0000","last_fullsized":"2023-11-04T05:28:25.834888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:10:25.496022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834773+0000","last_change":"2023-11-04T05:28:21.456943+0000","last_active":"2023-11-04T05:28:25.834773+0000","last_peered":"2023-11-04T05:28:25.834773+0000","last_clean":"2023-11-04T05:28:25.834773+0000","last_became_active":"2023-11-04T05:28:21.456658+0000","last_became_peered":"2023-11-04T05:28:21.456658+0000","last_unstale":"2023-11-04T05:28:25.834773+0000","last_undegraded":"2023-11-04T05:28:25.834773+0000","last_fullsized":"2023-11-04T05:28:25.834773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:00:47.803141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.838509+0000","last_change":"2023-11-04T05:28:25.865380+0000","last_active":"2023-11-04T05:28:26.838509+0000","last_peered":"2023-11-04T05:28:26.838509+0000","last_clean":"2023-11-04T05:28:26.838509+0000","last_became_active":"2023-11-04T05:28:25.865014+0000","last_became_peered":"2023-11-04T05:28:25.865014+0000","last_unstale":"2023-11-04T05:28:26.838509+0000","last_undegraded":"2023-11-04T05:28:26.838509+0000","last_fullsized":"2023-11-04T05:28:26.838509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:08.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846470+0000","last_change":"2023-11-04T05:28:20.451801+0000","last_active":"2023-11-04T05:28:25.846470+0000","last_peered":"2023-11-04T05:28:25.846470+0000","last_clean":"2023-11-04T05:28:25.846470+0000","last_became_active":"2023-11-04T05:28:20.451470+0000","last_became_peered":"2023-11-04T05:28:20.451470+0000","last_unstale":"2023-11-04T05:28:25.846470+0000","last_undegraded":"2023-11-04T05:28:25.846470+0000","last_fullsized":"2023-11-04T05:28:25.846470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:26:53.409369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834801+0000","last_change":"2023-11-04T05:28:21.465901+0000","last_active":"2023-11-04T05:28:25.834801+0000","last_peered":"2023-11-04T05:28:25.834801+0000","last_clean":"2023-11-04T05:28:25.834801+0000","last_became_active":"2023-11-04T05:28:21.465612+0000","last_became_peered":"2023-11-04T05:28:21.465612+0000","last_unstale":"2023-11-04T05:28:25.834801+0000","last_undegraded":"2023-11-04T05:28:25.834801+0000","last_fullsized":"2023-11-04T05:28:25.834801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:58:25.911728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:27.919553+0000","last_change":"2023-11-04T05:28:25.858664+0000","last_active":"2023-11-04T05:28:27.919553+0000","last_peered":"2023-11-04T05:28:27.919553+0000","last_clean":"2023-11-04T05:28:27.919553+0000","last_became_active":"2023-11-04T05:28:25.858208+0000","last_became_peered":"2023-11-04T05:28:25.858208+0000","last_unstale":"2023-11-04T05:28:27.919553+0000","last_undegraded":"2023-11-04T05:28:27.919553+0000","last_fullsized":"2023-11-04T05:28:27.919553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:41:26.004032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858515+0000","last_change":"2023-11-04T05:28:20.443824+0000","last_active":"2023-11-04T05:28:25.858515+0000","last_peered":"2023-11-04T05:28:25.858515+0000","last_clean":"2023-11-04T05:28:25.858515+0000","last_became_active":"2023-11-04T05:28:20.443479+0000","last_became_peered":"2023-11-04T05:28:20.443479+0000","last_unstale":"2023-11-04T05:28:25.858515+0000","last_undegraded":"2023-11-04T05:28:25.858515+0000","last_fullsized":"2023-11-04T05:28:25.858515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:21:43.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840613+0000","last_change":"2023-11-04T05:28:21.463699+0000","last_active":"2023-11-04T05:28:25.840613+0000","last_peered":"2023-11-04T05:28:25.840613+0000","last_clean":"2023-11-04T05:28:25.840613+0000","last_became_active":"2023-11-04T05:28:21.462308+0000","last_became_peered":"2023-11-04T05:28:21.462308+0000","last_unstale":"2023-11-04T05:28:25.840613+0000","last_undegraded":"2023-11-04T05:28:25.840613+0000","last_fullsized":"2023-11-04T05:28:25.840613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:41:23.610136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-04T05:28:26.836473+0000","last_change":"2023-11-04T05:28:25.849562+0000","last_active":"2023-11-04T05:28:26.836473+0000","last_peered":"2023-11-04T05:28:26.836473+0000","last_clean":"2023-11-04T05:28:26.836473+0000","last_became_active":"2023-11-04T05:28:25.849142+0000","last_became_peered":"2023-11-04T05:28:25.849142+0000","last_unstale":"2023-11-04T05:28:26.836473+0000","last_undegraded":"2023-11-04T05:28:26.836473+0000","last_fullsized":"2023-11-04T05:28:26.836473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:24.818920+0000","last_clean_scrub_stamp":"2023-11-04T05:28:24.818920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:12:55.575788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.846568+0000","last_change":"2023-11-04T05:28:20.455376+0000","last_active":"2023-11-04T05:28:25.846568+0000","last_peered":"2023-11-04T05:28:25.846568+0000","last_clean":"2023-11-04T05:28:25.846568+0000","last_became_active":"2023-11-04T05:28:20.455069+0000","last_became_peered":"2023-11-04T05:28:20.455069+0000","last_unstale":"2023-11-04T05:28:25.846568+0000","last_undegraded":"2023-11-04T05:28:25.846568+0000","last_fullsized":"2023-11-04T05:28:25.846568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:19.418011+0000","last_clean_scrub_stamp":"2023-11-04T05:28:19.418011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:26:44.083516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835570+0000","last_change":"2023-11-04T05:28:21.459762+0000","last_active":"2023-11-04T05:28:25.835570+0000","last_peered":"2023-11-04T05:28:25.835570+0000","last_clean":"2023-11-04T05:28:25.835570+0000","last_became_active":"2023-11-04T05:28:21.459488+0000","last_became_peered":"2023-11-04T05:28:21.459488+0000","last_unstale":"2023-11-04T05:28:25.835570+0000","last_undegraded":"2023-11-04T05:28:25.835570+0000","last_fullsized":"2023-11-04T05:28:25.835570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:40:37.472151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.836085+0000","last_change":"2023-11-04T05:28:21.458229+0000","last_active":"2023-11-04T05:28:25.836085+0000","last_peered":"2023-11-04T05:28:25.836085+0000","last_clean":"2023-11-04T05:28:25.836085+0000","last_became_active":"2023-11-04T05:28:21.457918+0000","last_became_peered":"2023-11-04T05:28:21.457918+0000","last_unstale":"2023-11-04T05:28:25.836085+0000","last_undegraded":"2023-11-04T05:28:25.836085+0000","last_fullsized":"2023-11-04T05:28:25.836085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:42:17.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.845334+0000","last_change":"2023-11-04T05:28:21.464900+0000","last_active":"2023-11-04T05:28:25.845334+0000","last_peered":"2023-11-04T05:28:25.845334+0000","last_clean":"2023-11-04T05:28:25.845334+0000","last_became_active":"2023-11-04T05:28:21.463859+0000","last_became_peered":"2023-11-04T05:28:21.463859+0000","last_unstale":"2023-11-04T05:28:25.845334+0000","last_undegraded":"2023-11-04T05:28:25.845334+0000","last_fullsized":"2023-11-04T05:28:25.845334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:07.946555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839310+0000","last_change":"2023-11-04T05:28:21.466114+0000","last_active":"2023-11-04T05:28:25.839310+0000","last_peered":"2023-11-04T05:28:25.839310+0000","last_clean":"2023-11-04T05:28:25.839310+0000","last_became_active":"2023-11-04T05:28:21.465547+0000","last_became_peered":"2023-11-04T05:28:21.465547+0000","last_unstale":"2023-11-04T05:28:25.839310+0000","last_undegraded":"2023-11-04T05:28:25.839310+0000","last_fullsized":"2023-11-04T05:28:25.839310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:15:04.065689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.842759+0000","last_change":"2023-11-04T05:28:21.456527+0000","last_active":"2023-11-04T05:28:25.842759+0000","last_peered":"2023-11-04T05:28:25.842759+0000","last_clean":"2023-11-04T05:28:25.842759+0000","last_became_active":"2023-11-04T05:28:21.456157+0000","last_became_peered":"2023-11-04T05:28:21.456157+0000","last_unstale":"2023-11-04T05:28:25.842759+0000","last_undegraded":"2023-11-04T05:28:25.842759+0000","last_fullsized":"2023-11-04T05:28:25.842759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:26:19.149382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.840466+0000","last_change":"2023-11-04T05:28:21.460439+0000","last_active":"2023-11-04T05:28:25.840466+0000","last_peered":"2023-11-04T05:28:25.840466+0000","last_clean":"2023-11-04T05:28:25.840466+0000","last_became_active":"2023-11-04T05:28:21.460146+0000","last_became_peered":"2023-11-04T05:28:21.460146+0000","last_unstale":"2023-11-04T05:28:25.840466+0000","last_undegraded":"2023-11-04T05:28:25.840466+0000","last_fullsized":"2023-11-04T05:28:25.840466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:40:21.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.858719+0000","last_change":"2023-11-04T05:28:21.447651+0000","last_active":"2023-11-04T05:28:25.858719+0000","last_peered":"2023-11-04T05:28:25.858719+0000","last_clean":"2023-11-04T05:28:25.858719+0000","last_became_active":"2023-11-04T05:28:21.447381+0000","last_became_peered":"2023-11-04T05:28:21.447381+0000","last_unstale":"2023-11-04T05:28:25.858719+0000","last_undegraded":"2023-11-04T05:28:25.858719+0000","last_fullsized":"2023-11-04T05:28:25.858719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.835914+0000","last_change":"2023-11-04T05:28:21.443196+0000","last_active":"2023-11-04T05:28:25.835914+0000","last_peered":"2023-11-04T05:28:25.835914+0000","last_clean":"2023-11-04T05:28:25.835914+0000","last_became_active":"2023-11-04T05:28:21.442860+0000","last_became_peered":"2023-11-04T05:28:21.442860+0000","last_unstale":"2023-11-04T05:28:25.835914+0000","last_undegraded":"2023-11-04T05:28:25.835914+0000","last_fullsized":"2023-11-04T05:28:25.835914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:40:48.937037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841180+0000","last_change":"2023-11-04T05:28:21.455732+0000","last_active":"2023-11-04T05:28:25.841180+0000","last_peered":"2023-11-04T05:28:25.841180+0000","last_clean":"2023-11-04T05:28:25.841180+0000","last_became_active":"2023-11-04T05:28:21.455434+0000","last_became_peered":"2023-11-04T05:28:21.455434+0000","last_unstale":"2023-11-04T05:28:25.841180+0000","last_undegraded":"2023-11-04T05:28:25.841180+0000","last_fullsized":"2023-11-04T05:28:25.841180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839745+0000","last_change":"2023-11-04T05:28:21.457789+0000","last_active":"2023-11-04T05:28:25.839745+0000","last_peered":"2023-11-04T05:28:25.839745+0000","last_clean":"2023-11-04T05:28:25.839745+0000","last_became_active":"2023-11-04T05:28:21.457521+0000","last_became_peered":"2023-11-04T05:28:21.457521+0000","last_unstale":"2023-11-04T05:28:25.839745+0000","last_undegraded":"2023-11-04T05:28:25.839745+0000","last_fullsized":"2023-11-04T05:28:25.839745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:52:30.125679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.841996+0000","last_change":"2023-11-04T05:28:21.462501+0000","last_active":"2023-11-04T05:28:25.841996+0000","last_peered":"2023-11-04T05:28:25.841996+0000","last_clean":"2023-11-04T05:28:25.841996+0000","last_became_active":"2023-11-04T05:28:21.462147+0000","last_became_peered":"2023-11-04T05:28:21.462147+0000","last_unstale":"2023-11-04T05:28:25.841996+0000","last_undegraded":"2023-11-04T05:28:25.841996+0000","last_fullsized":"2023-11-04T05:28:25.841996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:10:20.871478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.839367+0000","last_change":"2023-11-04T05:28:21.462044+0000","last_active":"2023-11-04T05:28:25.839367+0000","last_peered":"2023-11-04T05:28:25.839367+0000","last_clean":"2023-11-04T05:28:25.839367+0000","last_became_active":"2023-11-04T05:28:21.461816+0000","last_became_peered":"2023-11-04T05:28:21.461816+0000","last_unstale":"2023-11-04T05:28:25.839367+0000","last_undegraded":"2023-11-04T05:28:25.839367+0000","last_fullsized":"2023-11-04T05:28:25.839367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:03:40.402680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.843103+0000","last_change":"2023-11-04T05:28:21.458256+0000","last_active":"2023-11-04T05:28:25.843103+0000","last_peered":"2023-11-04T05:28:25.843103+0000","last_clean":"2023-11-04T05:28:25.843103+0000","last_became_active":"2023-11-04T05:28:21.457944+0000","last_became_peered":"2023-11-04T05:28:21.457944+0000","last_unstale":"2023-11-04T05:28:25.843103+0000","last_undegraded":"2023-11-04T05:28:25.843103+0000","last_fullsized":"2023-11-04T05:28:25.843103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:41:28.397824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-04T05:28:25.834927+0000","last_change":"2023-11-04T05:28:21.451553+0000","last_active":"2023-11-04T05:28:25.834927+0000","last_peered":"2023-11-04T05:28:25.834927+0000","last_clean":"2023-11-04T05:28:25.834927+0000","last_became_active":"2023-11-04T05:28:21.451276+0000","last_became_peered":"2023-11-04T05:28:21.451276+0000","last_unstale":"2023-11-04T05:28:25.834927+0000","last_undegraded":"2023-11-04T05:28:25.834927+0000","last_fullsized":"2023-11-04T05:28:25.834927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:20.423139+0000","last_clean_scrub_stamp":"2023-11-04T05:28:20.423139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:59:25.789338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":697852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":697908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.958}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.939}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":107028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542156,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":110744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":106992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-04T05:28:51.335 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-04T05:28:51.336 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-04T05:28:51.336 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-04T05:28:51.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-04T05:28:51.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:28:51.459 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:28:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.491+0000 7f223c719700 1 Processor -- start 2023-11-04T05:28:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.491+0000 7f223c719700 1 -- start start 2023-11-04T05:28:51.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f223c719700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f22341526c0 0x7f22340b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f223c719700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2234152aa0 0x7f22340cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f223c719700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223415c250 con 0x7f223415bd70 2023-11-04T05:28:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f223c719700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223415cdd0 con 0x7f2234152aa0 2023-11-04T05:28:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f223c719700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223415c530 con 0x7f22341526c0 2023-11-04T05:28:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2234152aa0 0x7f22340cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2234152aa0 0x7f22340cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52364/0 (socket says 172.21.15.3:52364) 2023-11-04T05:28:51.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 -- 172.21.15.3:0/3527817126 learned_addr learned my addr 172.21.15.3:0/3527817126 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f223a4b5700 1 --2- 172.21.15.3:0/3527817126 >> v2:172.21.15.188:3300/0 conn(0x7f22341526c0 0x7f22340b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 -- 172.21.15.3:0/3527817126 >> v2:172.21.15.188:3300/0 conn(0x7f22341526c0 msgr2=0x7f22340b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 --2- 172.21.15.3:0/3527817126 >> v2:172.21.15.188:3300/0 conn(0x7f22341526c0 0x7f22340b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 -- 172.21.15.3:0/3527817126 >> v1:172.21.15.188:6789/0 conn(0x7f223415bd70 legacy=0x7f22340cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 -- 172.21.15.3:0/3527817126 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223415c810 con 0x7f2234152aa0 2023-11-04T05:28:51.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f2239cb4700 1 --2- 172.21.15.3:0/3527817126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234152aa0 0x7f22340cde20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22340d2700 tx=0x7f2224009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841c242496a3731b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.493+0000 7f22394b3700 1 -- 172.21.15.3:0/3527817126 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224011830 con 0x7f2234152aa0 2023-11-04T05:28:51.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f22394b3700 1 -- 172.21.15.3:0/3527817126 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224011990 con 0x7f2234152aa0 2023-11-04T05:28:51.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f22394b3700 1 -- 172.21.15.3:0/3527817126 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224011d20 con 0x7f2234152aa0 2023-11-04T05:28:51.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 -- 172.21.15.3:0/3527817126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234152aa0 msgr2=0x7f22340cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 --2- 172.21.15.3:0/3527817126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234152aa0 0x7f22340cde20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22340d2700 tx=0x7f2224009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 -- 172.21.15.3:0/3527817126 shutdown_connections 2023-11-04T05:28:51.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 --2- 172.21.15.3:0/3527817126 >> v2:172.21.15.188:3300/0 conn(0x7f22341526c0 0x7f22340b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 --2- 172.21.15.3:0/3527817126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234152aa0 0x7f22340cde20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 -- 172.21.15.3:0/3527817126 >> 172.21.15.3:0/3527817126 conn(0x7f223406ab20 msgr2=0x7f223406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:51.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 -- 172.21.15.3:0/3527817126 shutdown_connections 2023-11-04T05:28:51.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.494+0000 7f223c719700 1 -- 172.21.15.3:0/3527817126 wait complete. 2023-11-04T05:28:51.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.495+0000 7f223c719700 1 Processor -- start 2023-11-04T05:28:51.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.495+0000 7f223c719700 1 -- start start 2023-11-04T05:28:51.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223c719700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 0x7f223414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223c719700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223415bd70 0x7f22340cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223c719700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22340c7820 0x7f2234153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223c719700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c70e0 con 0x7f22341526c0 2023-11-04T05:28:51.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223c719700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340d00c0 con 0x7f223415bd70 2023-11-04T05:28:51.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223c719700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340d1370 con 0x7f22340c7820 2023-11-04T05:28:51.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223a4b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 0x7f223414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223a4b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 0x7f223414d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52368/0 (socket says 172.21.15.3:52368) 2023-11-04T05:28:51.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223a4b5700 1 -- 172.21.15.3:0/2303764924 learned_addr learned my addr 172.21.15.3:0/2303764924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:51.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f2239cb4700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223415bd70 0x7f22340cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223acb6700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22340c7820 0x7f2234153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223a4b5700 1 -- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22340c7820 msgr2=0x7f2234153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.496+0000 7f223a4b5700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22340c7820 0x7f2234153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f223a4b5700 1 -- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223415bd70 msgr2=0x7f22340cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f223a4b5700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223415bd70 0x7f22340cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f223a4b5700 1 -- 172.21.15.3:0/2303764924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009550 con 0x7f22341526c0 2023-11-04T05:28:51.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f223a4b5700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 0x7f223414d900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9b0 tx=0x7f223000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230009710 con 0x7f22341526c0 2023-11-04T05:28:51.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000ee50 con 0x7f22341526c0 2023-11-04T05:28:51.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22340d02d0 con 0x7f22341526c0 2023-11-04T05:28:51.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000ca50 con 0x7f22341526c0 2023-11-04T05:28:51.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.497+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22340d0890 con 0x7f22341526c0 2023-11-04T05:28:51.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.498+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2234074df0 con 0x7f22341526c0 2023-11-04T05:28:51.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.498+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f223000cbb0 con 0x7f22341526c0 2023-11-04T05:28:51.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.499+0000 7f222affd700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f221806af00 0x7f221806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.499+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f22300898e0 con 0x7f22341526c0 2023-11-04T05:28:51.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.499+0000 7f2239cb4700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f221806af00 0x7f221806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.500+0000 7f2239cb4700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f221806af00 0x7f221806d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2224009c10 tx=0x7f2224008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.503+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f22300546a0 con 0x7f22341526c0 2023-11-04T05:28:51.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.751+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f22340b3e80 con 0x7f22341526c0 2023-11-04T05:28:51.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.751+0000 7f222affd700 1 -- 172.21.15.3:0/2303764924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f223000ce60 con 0x7f22341526c0 2023-11-04T05:28:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:51.754 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-04T05:28:51.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f221806af00 msgr2=0x7f221806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f221806af00 0x7f221806d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2224009c10 tx=0x7f2224008030 comp rx=0 tx=0).stop 2023-11-04T05:28:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 msgr2=0x7f223414d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 0x7f223414d900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9b0 tx=0x7f223000ecc0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 shutdown_connections 2023-11-04T05:28:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f221806af00 0x7f221806d3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22340c7820 0x7f2234153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22341526c0 0x7f223414d900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 --2- 172.21.15.3:0/2303764924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223415bd70 0x7f22340cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 >> 172.21.15.3:0/2303764924 conn(0x7f223406ab20 msgr2=0x7f22340b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:51.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 shutdown_connections 2023-11-04T05:28:51.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.753+0000 7f223c719700 1 -- 172.21.15.3:0/2303764924 wait complete. 2023-11-04T05:28:51.767 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-04T05:28:51.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.891+0000 7fa1594aa700 1 Processor -- start 2023-11-04T05:28:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.891+0000 7fa1594aa700 1 -- start start 2023-11-04T05:28:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.892+0000 7fa1594aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 0x7fa1540cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.892+0000 7fa1594aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa154154930 con 0x7fa15406d790 2023-11-04T05:28:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.892+0000 7fa1594aa700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15415e9b0 con 0x7fa15415e350 2023-11-04T05:28:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.892+0000 7fa1594aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540cddf0 con 0x7fa154154550 2023-11-04T05:28:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.892+0000 7fa1527fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa15406d790 0x7fa15415dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57724/0 (socket says 172.21.15.3:57724) 2023-11-04T05:28:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.892+0000 7fa1527fc700 1 -- 172.21.15.3:0/941090178 learned_addr learned my addr 172.21.15.3:0/941090178 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa152ffd700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 0x7fa1540cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038211718 0 0) 0x7fa154154930 con 0x7fa15406d790 2023-11-04T05:28:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa13c0034d0 con 0x7fa15406d790 2023-11-04T05:28:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731257250 0 0) 0x7fa13c0034d0 con 0x7fa15406d790 2023-11-04T05:28:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 msgr2=0x7fa1540cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 0x7fa1540cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 >> v1:172.21.15.188:6789/0 conn(0x7fa15415e350 legacy=0x7fa15406ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540ce0d0 con 0x7fa15406d790 2023-11-04T05:28:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fa1440017f0 con 0x7fa15406d790 2023-11-04T05:28:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 >> v1:172.21.15.3:6789/0 conn(0x7fa15406d790 legacy=0x7fa15415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c003b20 0x7fa13c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa13c00acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 0x7fa13c00d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540cddf0 con 0x7fa13c0088e0 2023-11-04T05:28:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00dd40 con 0x7fa13c003b20 2023-11-04T05:28:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00df00 con 0x7fa154154550 2023-11-04T05:28:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1527fc700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa13c00acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa152ffd700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c003b20 0x7fa13c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1537fe700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 0x7fa13c00d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1527fc700 1 -- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 msgr2=0x7fa13c00d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1527fc700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa154154550 0x7fa13c00d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1527fc700 1 -- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c003b20 msgr2=0x7fa13c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1527fc700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c003b20 0x7fa13c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.893+0000 7fa1527fc700 1 -- 172.21.15.3:0/941090178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0034d0 con 0x7fa13c0088e0 2023-11-04T05:28:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1527fc700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa13c00acf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa14400c240 tx=0x7fa13c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97275bdad5a14ee server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400cde0 con 0x7fa13c0088e0 2023-11-04T05:28:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144007b80 con 0x7fa13c0088e0 2023-11-04T05:28:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa151ffb700 1 -- 172.21.15.3:0/941090178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1440072f0 con 0x7fa13c0088e0 2023-11-04T05:28:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1594aa700 1 -- 172.21.15.3:0/941090178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 msgr2=0x7fa13c00acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1594aa700 1 --2- 172.21.15.3:0/941090178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa13c00acf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa14400c240 tx=0x7fa13c003780 comp rx=0 tx=0).stop 2023-11-04T05:28:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1537fe700 1 -- 172.21.15.3:0/941090178 reap_dead start 2023-11-04T05:28:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1594aa700 1 -- 172.21.15.3:0/941090178 shutdown_connections 2023-11-04T05:28:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1594aa700 1 -- 172.21.15.3:0/941090178 >> 172.21.15.3:0/941090178 conn(0x7fa1540cfd50 msgr2=0x7fa1540d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1594aa700 1 -- 172.21.15.3:0/941090178 shutdown_connections 2023-11-04T05:28:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.894+0000 7fa1594aa700 1 -- 172.21.15.3:0/941090178 wait complete. 2023-11-04T05:28:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.896+0000 7fa1594aa700 1 Processor -- start 2023-11-04T05:28:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.896+0000 7fa1594aa700 1 -- start start 2023-11-04T05:28:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa1541f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1541f0600 0x7fa1540c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1540c9c00 0x7fa1541f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15414dd40 con 0x7fa13c0088e0 2023-11-04T05:28:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540ca110 con 0x7fa1540c9c00 2023-11-04T05:28:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15414d410 con 0x7fa1541f0600 2023-11-04T05:28:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa1541f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa1541f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52388/0 (socket says 172.21.15.3:52388) 2023-11-04T05:28:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 -- 172.21.15.3:0/1922975418 learned_addr learned my addr 172.21.15.3:0/1922975418 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1537fe700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1540c9c00 0x7fa1541f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1527fc700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1541f0600 0x7fa1540c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 -- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1541f0600 msgr2=0x7fa1540c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1541f0600 0x7fa1540c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 -- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1540c9c00 msgr2=0x7fa1541f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1540c9c00 0x7fa1541f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 -- 172.21.15.3:0/1922975418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15414d6f0 con 0x7fa13c0088e0 2023-11-04T05:28:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa152ffd700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa1541f23a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa148011aa0 tx=0x7fa148011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800f980 con 0x7fa13c0088e0 2023-11-04T05:28:51.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14800fae0 con 0x7fa13c0088e0 2023-11-04T05:28:51.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15414d9d0 con 0x7fa13c0088e0 2023-11-04T05:28:51.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.897+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800fe90 con 0x7fa13c0088e0 2023-11-04T05:28:51.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.898+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1541f0a80 con 0x7fa13c0088e0 2023-11-04T05:28:51.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.898+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15414e020 con 0x7fa13c0088e0 2023-11-04T05:28:51.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.899+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa148014440 con 0x7fa13c0088e0 2023-11-04T05:28:51.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.899+0000 7fa13b7fe700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa14006af00 0x7fa14006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:51.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.899+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa14808a0b0 con 0x7fa13c0088e0 2023-11-04T05:28:51.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.899+0000 7fa1527fc700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa14006af00 0x7fa14006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:51.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.900+0000 7fa1527fc700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa14006af00 0x7fa14006d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa144001ab0 tx=0x7fa144007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:51.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:51.904+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa148054f20 con 0x7fa13c0088e0 2023-11-04T05:28:52.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.107+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa154074df0 con 0x7fa13c0088e0 2023-11-04T05:28:52.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.108+0000 7fa13b7fe700 1 -- 172.21.15.3:0/1922975418 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7fa148058540 con 0x7fa13c0088e0 2023-11-04T05:28:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:52.111 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:52.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa14006af00 msgr2=0x7fa14006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa14006af00 0x7fa14006d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa144001ab0 tx=0x7fa144007000 comp rx=0 tx=0).stop 2023-11-04T05:28:52.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 msgr2=0x7fa1541f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa1541f23a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa148011aa0 tx=0x7fa148011e60 comp rx=0 tx=0).stop 2023-11-04T05:28:52.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 shutdown_connections 2023-11-04T05:28:52.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa14006af00 0x7fa14006d3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1540c9c00 0x7fa1541f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa13c0088e0 0x7fa1541f23a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 --2- 172.21.15.3:0/1922975418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1541f0600 0x7fa1540c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 >> 172.21.15.3:0/1922975418 conn(0x7fa1540cfd50 msgr2=0x7fa1540cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:52.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 shutdown_connections 2023-11-04T05:28:52.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.110+0000 7fa1594aa700 1 -- 172.21.15.3:0/1922975418 wait complete. 2023-11-04T05:28:52.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:28:52.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:28:52.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.251+0000 7f6efe791700 1 Processor -- start 2023-11-04T05:28:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.251+0000 7f6efe791700 1 -- start start 2023-11-04T05:28:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.252+0000 7f6efe791700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ef80ceaa0 0x7f6ef80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.252+0000 7f6efe791700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef815b870 con 0x7f6ef80cee80 2023-11-04T05:28:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6efe791700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef80ccc70 con 0x7f6ef815b430 2023-11-04T05:28:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6efe791700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef80cdee0 con 0x7f6ef80ceaa0 2023-11-04T05:28:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ef80ceaa0 0x7f6ef80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef77fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ef80cee80 0x7f6ef815ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57744/0 (socket says 172.21.15.3:57744) 2023-11-04T05:28:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef77fe700 1 -- 172.21.15.3:0/4084347389 learned_addr learned my addr 172.21.15.3:0/4084347389 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef7fff700 1 -- 172.21.15.3:0/4084347389 >> v1:172.21.15.188:6789/0 conn(0x7f6ef815b430 legacy=0x7f6ef80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef7fff700 1 -- 172.21.15.3:0/4084347389 >> v1:172.21.15.3:6789/0 conn(0x7f6ef80cee80 legacy=0x7f6ef815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef7fff700 1 -- 172.21.15.3:0/4084347389 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef815baa0 con 0x7f6ef80ceaa0 2023-11-04T05:28:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef6ffd700 1 -- 172.21.15.3:0/4084347389 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463634007 0 0) 0x7f6ef815b870 con 0x7f6ef80cee80 2023-11-04T05:28:52.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef7fff700 1 --2- 172.21.15.3:0/4084347389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef80ceaa0 0x7f6ef80cd9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009840 tx=0x7f6ee8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19b87a44d61daf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef6ffd700 1 -- 172.21.15.3:0/4084347389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8011830 con 0x7f6ef80ceaa0 2023-11-04T05:28:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef6ffd700 1 -- 172.21.15.3:0/4084347389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee8011990 con 0x7f6ef80ceaa0 2023-11-04T05:28:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.253+0000 7f6ef6ffd700 1 -- 172.21.15.3:0/4084347389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8011d20 con 0x7f6ef80ceaa0 2023-11-04T05:28:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 -- 172.21.15.3:0/4084347389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef80ceaa0 msgr2=0x7f6ef80cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 --2- 172.21.15.3:0/4084347389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef80ceaa0 0x7f6ef80cd9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009840 tx=0x7f6ee8009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 -- 172.21.15.3:0/4084347389 shutdown_connections 2023-11-04T05:28:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 --2- 172.21.15.3:0/4084347389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef80ceaa0 0x7f6ef80cd9a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 -- 172.21.15.3:0/4084347389 >> 172.21.15.3:0/4084347389 conn(0x7f6ef8072560 msgr2=0x7f6ef8099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 -- 172.21.15.3:0/4084347389 shutdown_connections 2023-11-04T05:28:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.254+0000 7f6efe791700 1 -- 172.21.15.3:0/4084347389 wait complete. 2023-11-04T05:28:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.255+0000 7f6efe791700 1 Processor -- start 2023-11-04T05:28:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.255+0000 7f6efe791700 1 -- start start 2023-11-04T05:28:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efe791700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef80ceaa0 0x7f6ef8158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efe791700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef80cee80 0x7f6ef81ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efe791700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 0x7f6ef80b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efe791700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8159bb0 con 0x7f6ef815b430 2023-11-04T05:28:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efe791700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8159440 con 0x7f6ef80cee80 2023-11-04T05:28:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efe791700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81f0610 con 0x7f6ef80ceaa0 2023-11-04T05:28:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efcd2e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 0x7f6ef80b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efcd2e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 0x7f6ef80b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52406/0 (socket says 172.21.15.3:52406) 2023-11-04T05:28:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6efcd2e700 1 -- 172.21.15.3:0/1170079398 learned_addr learned my addr 172.21.15.3:0/1170079398 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6ef77fe700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef80cee80 0x7f6ef81ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.256+0000 7f6ef7fff700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef80ceaa0 0x7f6ef8158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efcd2e700 1 -- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef80ceaa0 msgr2=0x7f6ef8158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efcd2e700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef80ceaa0 0x7f6ef8158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efcd2e700 1 -- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef80cee80 msgr2=0x7f6ef81ed540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efcd2e700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef80cee80 0x7f6ef81ed540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efcd2e700 1 -- 172.21.15.3:0/1170079398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80094a0 con 0x7f6ef815b430 2023-11-04T05:28:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6ef77fe700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef80cee80 0x7f6ef81ed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efcd2e700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 0x7f6ef80b3e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000cb50 tx=0x7f6ef000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0003c00 con 0x7f6ef815b430 2023-11-04T05:28:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0003d60 con 0x7f6ef815b430 2023-11-04T05:28:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef81f08f0 con 0x7f6ef815b430 2023-11-04T05:28:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000ab70 con 0x7f6ef815b430 2023-11-04T05:28:52.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.257+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef81f0eb0 con 0x7f6ef815b430 2023-11-04T05:28:52.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.258+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef815d800 con 0x7f6ef815b430 2023-11-04T05:28:52.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.258+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ef000acd0 con 0x7f6ef815b430 2023-11-04T05:28:52.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.259+0000 7f6ef4ff9700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6ed806af00 0x7f6ed806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.259+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ef0089920 con 0x7f6ef815b430 2023-11-04T05:28:52.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.259+0000 7f6ef7fff700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6ed806af00 0x7f6ed806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.260+0000 7f6ef7fff700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6ed806af00 0x7f6ed806d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80090b0 tx=0x7f6ee8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.263+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f6ef00546e0 con 0x7f6ef815b430 2023-11-04T05:28:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.466+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ef81eda80 con 0x7f6ef815b430 2023-11-04T05:28:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.466+0000 7f6ef4ff9700 1 -- 172.21.15.3:0/1170079398 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ef0057d00 con 0x7f6ef815b430 2023-11-04T05:28:52.469 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:28:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6ed806af00 msgr2=0x7f6ed806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6ed806af00 0x7f6ed806d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80090b0 tx=0x7f6ee8008030 comp rx=0 tx=0).stop 2023-11-04T05:28:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 msgr2=0x7f6ef80b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 0x7f6ef80b3e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000cb50 tx=0x7f6ef000cf10 comp rx=0 tx=0).stop 2023-11-04T05:28:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 shutdown_connections 2023-11-04T05:28:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6ed806af00 0x7f6ed806d3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ef815b430 0x7f6ef80b3e30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef80ceaa0 0x7f6ef8158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 --2- 172.21.15.3:0/1170079398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef80cee80 0x7f6ef81ed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 >> 172.21.15.3:0/1170079398 conn(0x7f6ef8072560 msgr2=0x7f6ef806e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 shutdown_connections 2023-11-04T05:28:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.469+0000 7f6efe791700 1 -- 172.21.15.3:0/1170079398 wait complete. 2023-11-04T05:28:52.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:28:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 Processor -- start 2023-11-04T05:28:52.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 -- start start 2023-11-04T05:28:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0e80ceaa0 0x7fd0e80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0e815b430 0x7fd0e80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e815b870 con 0x7fd0e80cee80 2023-11-04T05:28:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80ccc70 con 0x7fd0e80ceaa0 2023-11-04T05:28:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.610+0000 7fd0edbf4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80cdee0 con 0x7fd0e815b430 2023-11-04T05:28:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e77fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0e80ceaa0 0x7fd0e80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e6ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd0e80cee80 0x7fd0e815ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57760/0 (socket says 172.21.15.3:57760) 2023-11-04T05:28:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e6ffd700 1 -- 172.21.15.3:0/1281024200 learned_addr learned my addr 172.21.15.3:0/1281024200 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1281024200 >> v2:172.21.15.188:3300/0 conn(0x7fd0e815b430 msgr2=0x7fd0e80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e7fff700 1 --2- 172.21.15.3:0/1281024200 >> v2:172.21.15.188:3300/0 conn(0x7fd0e815b430 0x7fd0e80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e77fe700 1 --2- 172.21.15.3:0/1281024200 >> v2:172.21.15.188:3300/0 conn(0x7fd0e815b430 0x7fd0e80caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1281024200 >> v1:172.21.15.3:6789/0 conn(0x7fd0e80cee80 legacy=0x7fd0e815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e67fc700 1 -- 172.21.15.3:0/1281024200 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702738773 0 0) 0x7fd0e815b870 con 0x7fd0e80cee80 2023-11-04T05:28:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1281024200 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e815baa0 con 0x7fd0e80ceaa0 2023-11-04T05:28:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e77fe700 1 --2- 172.21.15.3:0/1281024200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80ceaa0 0x7fd0e80cd9a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cac20 tx=0x7fd0d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8b412ece9dfc98 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e67fc700 1 -- 172.21.15.3:0/1281024200 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8011830 con 0x7fd0e80ceaa0 2023-11-04T05:28:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e67fc700 1 -- 172.21.15.3:0/1281024200 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d8011990 con 0x7fd0e80ceaa0 2023-11-04T05:28:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.611+0000 7fd0e67fc700 1 -- 172.21.15.3:0/1281024200 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8011d20 con 0x7fd0e80ceaa0 2023-11-04T05:28:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1281024200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80ceaa0 msgr2=0x7fd0e80cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1281024200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80ceaa0 0x7fd0e80cd9a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cac20 tx=0x7fd0d8009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1281024200 shutdown_connections 2023-11-04T05:28:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1281024200 >> v2:172.21.15.188:3300/0 conn(0x7fd0e815b430 0x7fd0e80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1281024200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80ceaa0 0x7fd0e80cd9a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1281024200 >> 172.21.15.3:0/1281024200 conn(0x7fd0e8072560 msgr2=0x7fd0e8099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1281024200 shutdown_connections 2023-11-04T05:28:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.612+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1281024200 wait complete. 2023-11-04T05:28:52.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.613+0000 7fd0edbf4700 1 Processor -- start 2023-11-04T05:28:52.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.613+0000 7fd0edbf4700 1 -- start start 2023-11-04T05:28:52.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0edbf4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 0x7fd0e814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0edbf4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e815b430 0x7fd0e80c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0edbf4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e80c9e30 0x7fd0e80c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0edbf4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80c5e10 con 0x7fd0e80cee80 2023-11-04T05:28:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0edbf4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80ca2b0 con 0x7fd0e815b430 2023-11-04T05:28:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0edbf4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80c5190 con 0x7fd0e80c9e30 2023-11-04T05:28:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 0x7fd0e814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 0x7fd0e814e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52420/0 (socket says 172.21.15.3:52420) 2023-11-04T05:28:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1818924570 learned_addr learned my addr 172.21.15.3:0/1818924570 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e7fff700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e80c9e30 0x7fd0e80c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e6ffd700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e815b430 0x7fd0e80c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e80c9e30 msgr2=0x7fd0e80c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e80c9e30 0x7fd0e80c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e815b430 msgr2=0x7fd0e80c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e815b430 0x7fd0e80c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.614+0000 7fd0e77fe700 1 -- 172.21.15.3:0/1818924570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d80094a0 con 0x7fd0e80cee80 2023-11-04T05:28:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.615+0000 7fd0e77fe700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 0x7fd0e814e6f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000c00 tx=0x7fd0d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.615+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80045d0 con 0x7fd0e80cee80 2023-11-04T05:28:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.615+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d801f070 con 0x7fd0e80cee80 2023-11-04T05:28:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.615+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e80c5470 con 0x7fd0e80cee80 2023-11-04T05:28:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.615+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80038a0 con 0x7fd0e80cee80 2023-11-04T05:28:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.615+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80c5a30 con 0x7fd0e80cee80 2023-11-04T05:28:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.616+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e8074df0 con 0x7fd0e80cee80 2023-11-04T05:28:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.617+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0d8003a00 con 0x7fd0e80cee80 2023-11-04T05:28:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.617+0000 7fd0cffff700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd0d406af00 0x7fd0d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.617+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0d8021070 con 0x7fd0e80cee80 2023-11-04T05:28:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.618+0000 7fd0e6ffd700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd0d406af00 0x7fd0d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.618+0000 7fd0e6ffd700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd0d406af00 0x7fd0d406d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8157920 tx=0x7fd0dc007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.621+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd0d8056190 con 0x7fd0e80cee80 2023-11-04T05:28:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.821+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0e80cc200 con 0x7fd0e80cee80 2023-11-04T05:28:52.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.821+0000 7fd0cffff700 1 -- 172.21.15.3:0/1818924570 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0d8003cb0 con 0x7fd0e80cee80 2023-11-04T05:28:52.824 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd0d406af00 msgr2=0x7fd0d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd0d406af00 0x7fd0d406d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8157920 tx=0x7fd0dc007de0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 msgr2=0x7fd0e814e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 0x7fd0e814e6f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000c00 tx=0x7fd0d8004750 comp rx=0 tx=0).stop 2023-11-04T05:28:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 shutdown_connections 2023-11-04T05:28:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd0d406af00 0x7fd0d406d3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e80c9e30 0x7fd0e80c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e80cee80 0x7fd0e814e6f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 --2- 172.21.15.3:0/1818924570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e815b430 0x7fd0e80c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 >> 172.21.15.3:0/1818924570 conn(0x7fd0e8072560 msgr2=0x7fd0e806ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 shutdown_connections 2023-11-04T05:28:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.824+0000 7fd0edbf4700 1 -- 172.21.15.3:0/1818924570 wait complete. 2023-11-04T05:28:52.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:28:52.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.966+0000 7fa437d17700 1 Processor -- start 2023-11-04T05:28:52.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.966+0000 7fa437d17700 1 -- start start 2023-11-04T05:28:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa437d17700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43006ab20 0x7fa43006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa437d17700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4300caca0 con 0x7fa4300d02d0 2023-11-04T05:28:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa437d17700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4300cfe40 con 0x7fa4301611d0 2023-11-04T05:28:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa437d17700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4300ce0a0 con 0x7fa43006ab20 2023-11-04T05:28:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4352b2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa43006ab20 0x7fa43006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4362b4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa4300d02d0 0x7fa4300d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57776/0 (socket says 172.21.15.3:57776) 2023-11-04T05:28:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4362b4700 1 -- 172.21.15.3:0/3130953817 learned_addr learned my addr 172.21.15.3:0/3130953817 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4352b2700 1 -- 172.21.15.3:0/3130953817 >> v1:172.21.15.188:6789/0 conn(0x7fa4301611d0 legacy=0x7fa4300ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4352b2700 1 -- 172.21.15.3:0/3130953817 >> v1:172.21.15.3:6789/0 conn(0x7fa4300d02d0 legacy=0x7fa4300d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4352b2700 1 -- 172.21.15.3:0/3130953817 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4300cf850 con 0x7fa43006ab20 2023-11-04T05:28:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa434ab1700 1 -- 172.21.15.3:0/3130953817 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208881660 0 0) 0x7fa4300caca0 con 0x7fa4300d02d0 2023-11-04T05:28:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.967+0000 7fa4352b2700 1 --2- 172.21.15.3:0/3130953817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43006ab20 0x7fa43006afa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa420009840 tx=0x7fa420009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84364e3e01f33d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa434ab1700 1 -- 172.21.15.3:0/3130953817 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420011830 con 0x7fa43006ab20 2023-11-04T05:28:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa434ab1700 1 -- 172.21.15.3:0/3130953817 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa420011990 con 0x7fa43006ab20 2023-11-04T05:28:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa434ab1700 1 -- 172.21.15.3:0/3130953817 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420011d20 con 0x7fa43006ab20 2023-11-04T05:28:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 -- 172.21.15.3:0/3130953817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43006ab20 msgr2=0x7fa43006afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 --2- 172.21.15.3:0/3130953817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43006ab20 0x7fa43006afa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa420009840 tx=0x7fa420009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 -- 172.21.15.3:0/3130953817 shutdown_connections 2023-11-04T05:28:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 --2- 172.21.15.3:0/3130953817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa43006ab20 0x7fa43006afa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 -- 172.21.15.3:0/3130953817 >> 172.21.15.3:0/3130953817 conn(0x7fa430072560 msgr2=0x7fa430099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 -- 172.21.15.3:0/3130953817 shutdown_connections 2023-11-04T05:28:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.968+0000 7fa437d17700 1 -- 172.21.15.3:0/3130953817 wait complete. 2023-11-04T05:28:52.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.969+0000 7fa437d17700 1 Processor -- start 2023-11-04T05:28:52.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.969+0000 7fa437d17700 1 -- start start 2023-11-04T05:28:52.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa437d17700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa43006ab20 0x7fa4301538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa437d17700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4300d02d0 0x7fa43015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa437d17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 0x7fa4300ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa437d17700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430160350 con 0x7fa4301611d0 2023-11-04T05:28:52.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa437d17700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300c4880 con 0x7fa43006ab20 2023-11-04T05:28:52.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa437d17700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43015f7d0 con 0x7fa4300d02d0 2023-11-04T05:28:52.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa4362b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 0x7fa4300ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa4362b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 0x7fa4300ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52434/0 (socket says 172.21.15.3:52434) 2023-11-04T05:28:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa4362b4700 1 -- 172.21.15.3:0/2492892351 learned_addr learned my addr 172.21.15.3:0/2492892351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa435ab3700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa43006ab20 0x7fa4301538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.970+0000 7fa4352b2700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4300d02d0 0x7fa43015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4362b4700 1 -- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4300d02d0 msgr2=0x7fa43015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4362b4700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4300d02d0 0x7fa43015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4362b4700 1 -- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa43006ab20 msgr2=0x7fa4301538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:52.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4362b4700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa43006ab20 0x7fa4301538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:52.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4362b4700 1 -- 172.21.15.3:0/2492892351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200094a0 con 0x7fa4301611d0 2023-11-04T05:28:52.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4362b4700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 0x7fa4300ada30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa428002230 tx=0x7fa42800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42800aac0 con 0x7fa4301611d0 2023-11-04T05:28:52.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42800ac40 con 0x7fa4301611d0 2023-11-04T05:28:52.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43015fab0 con 0x7fa4301611d0 2023-11-04T05:28:52.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4280070d0 con 0x7fa4301611d0 2023-11-04T05:28:52.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.971+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa430160070 con 0x7fa4301611d0 2023-11-04T05:28:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.972+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4300c4a40 con 0x7fa4301611d0 2023-11-04T05:28:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.972+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa428007250 con 0x7fa4301611d0 2023-11-04T05:28:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.973+0000 7fa4267fc700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa41406af00 0x7fa41406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.973+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa428089650 con 0x7fa4301611d0 2023-11-04T05:28:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.973+0000 7fa435ab3700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa41406af00 0x7fa41406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.974+0000 7fa435ab3700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa41406af00 0x7fa41406d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa42c007fd0 tx=0x7fa42c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:52.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:52.977+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa428054410 con 0x7fa4301611d0 2023-11-04T05:28:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.177+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa430153e30 con 0x7fa4301611d0 2023-11-04T05:28:53.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.178+0000 7fa4267fc700 1 -- 172.21.15.3:0/2492892351 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa428057a30 con 0x7fa4301611d0 2023-11-04T05:28:53.180 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:28:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa41406af00 msgr2=0x7fa41406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa41406af00 0x7fa41406d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa42c007fd0 tx=0x7fa42c009040 comp rx=0 tx=0).stop 2023-11-04T05:28:53.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 msgr2=0x7fa4300ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 0x7fa4300ada30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa428002230 tx=0x7fa42800cf50 comp rx=0 tx=0).stop 2023-11-04T05:28:53.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 shutdown_connections 2023-11-04T05:28:53.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa41406af00 0x7fa41406d3c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa43006ab20 0x7fa4301538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4301611d0 0x7fa4300ada30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 --2- 172.21.15.3:0/2492892351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4300d02d0 0x7fa43015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 >> 172.21.15.3:0/2492892351 conn(0x7fa430072560 msgr2=0x7fa4300b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:53.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 shutdown_connections 2023-11-04T05:28:53.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.180+0000 7fa437d17700 1 -- 172.21.15.3:0/2492892351 wait complete. 2023-11-04T05:28:53.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:28:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.319+0000 7faf48085700 1 Processor -- start 2023-11-04T05:28:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.319+0000 7faf48085700 1 -- start start 2023-11-04T05:28:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf48085700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf401611d0 0x7faf400ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf48085700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf4006b230 0x7faf400d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf48085700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf400caca0 con 0x7faf400cd4b0 2023-11-04T05:28:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf48085700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf400ce0a0 con 0x7faf4006b230 2023-11-04T05:28:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf48085700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf400cfe40 con 0x7faf401611d0 2023-11-04T05:28:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf45620700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faf400cd4b0 0x7faf4006ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57786/0 (socket says 172.21.15.3:57786) 2023-11-04T05:28:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf45620700 1 -- 172.21.15.3:0/1269107112 learned_addr learned my addr 172.21.15.3:0/1269107112 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf46622700 1 --2- 172.21.15.3:0/1269107112 >> v2:172.21.15.188:3300/0 conn(0x7faf4006b230 0x7faf400d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf45e21700 1 --2- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf401611d0 0x7faf400ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf44e1f700 1 -- 172.21.15.3:0/1269107112 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215080410 0 0) 0x7faf400caca0 con 0x7faf400cd4b0 2023-11-04T05:28:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.320+0000 7faf44e1f700 1 -- 172.21.15.3:0/1269107112 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf2c0035b0 con 0x7faf400cd4b0 2023-11-04T05:28:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf44e1f700 1 -- 172.21.15.3:0/1269107112 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801379600 0 0) 0x7faf2c0035b0 con 0x7faf400cd4b0 2023-11-04T05:28:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf46622700 1 -- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf401611d0 msgr2=0x7faf400ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf46622700 1 --2- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf401611d0 0x7faf400ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf46622700 1 -- 172.21.15.3:0/1269107112 >> v1:172.21.15.3:6789/0 conn(0x7faf400cd4b0 legacy=0x7faf4006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf46622700 1 -- 172.21.15.3:0/1269107112 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400cf850 con 0x7faf4006b230 2023-11-04T05:28:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf46622700 1 --2- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4006b230 0x7faf400d02d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf38009d90 tx=0x7faf3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7883133268b827f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf44e1f700 1 -- 172.21.15.3:0/1269107112 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3800e070 con 0x7faf4006b230 2023-11-04T05:28:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf44e1f700 1 -- 172.21.15.3:0/1269107112 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38003a20 con 0x7faf4006b230 2023-11-04T05:28:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.321+0000 7faf44e1f700 1 -- 172.21.15.3:0/1269107112 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c001670 con 0x7faf4006b230 2023-11-04T05:28:53.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 -- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4006b230 msgr2=0x7faf400d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 --2- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4006b230 0x7faf400d02d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf38009d90 tx=0x7faf3800b330 comp rx=0 tx=0).stop 2023-11-04T05:28:53.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 -- 172.21.15.3:0/1269107112 shutdown_connections 2023-11-04T05:28:53.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 --2- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf4006b230 0x7faf400d02d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 --2- 172.21.15.3:0/1269107112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf401611d0 0x7faf400ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 -- 172.21.15.3:0/1269107112 >> 172.21.15.3:0/1269107112 conn(0x7faf40072560 msgr2=0x7faf40099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 -- 172.21.15.3:0/1269107112 shutdown_connections 2023-11-04T05:28:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.322+0000 7faf48085700 1 -- 172.21.15.3:0/1269107112 wait complete. 2023-11-04T05:28:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.323+0000 7faf48085700 1 Processor -- start 2023-11-04T05:28:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.323+0000 7faf48085700 1 -- start start 2023-11-04T05:28:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf48085700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4006b230 0x7faf401536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf48085700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 0x7faf400aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf48085700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf401611d0 0x7faf400afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf48085700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400ae3b0 con 0x7faf400cd4b0 2023-11-04T05:28:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf48085700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400ad7a0 con 0x7faf401611d0 2023-11-04T05:28:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf48085700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf400ada80 con 0x7faf4006b230 2023-11-04T05:28:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 0x7faf400aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 0x7faf400aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52446/0 (socket says 172.21.15.3:52446) 2023-11-04T05:28:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 -- 172.21.15.3:0/3616992986 learned_addr learned my addr 172.21.15.3:0/3616992986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45e21700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4006b230 0x7faf401536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf46622700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf401611d0 0x7faf400afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 -- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4006b230 msgr2=0x7faf401536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4006b230 0x7faf401536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 -- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf401611d0 msgr2=0x7faf400afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf401611d0 0x7faf400afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 -- 172.21.15.3:0/3616992986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf38009a70 con 0x7faf400cd4b0 2023-11-04T05:28:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45e21700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4006b230 0x7faf401536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf46622700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf401611d0 0x7faf400afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf45620700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 0x7faf400aeb20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0011e0 tx=0x7faf3c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.324+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c007560 con 0x7faf400cd4b0 2023-11-04T05:28:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.325+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0076c0 con 0x7faf400cd4b0 2023-11-04T05:28:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.325+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf400add60 con 0x7faf400cd4b0 2023-11-04T05:28:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.325+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c004a90 con 0x7faf400cd4b0 2023-11-04T05:28:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.325+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf40154030 con 0x7faf400cd4b0 2023-11-04T05:28:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.325+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf40074df0 con 0x7faf400cd4b0 2023-11-04T05:28:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.326+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf3c004bf0 con 0x7faf400cd4b0 2023-11-04T05:28:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.326+0000 7faf367fc700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faf2406af00 0x7faf2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.326+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf3c088d70 con 0x7faf400cd4b0 2023-11-04T05:28:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.327+0000 7faf45e21700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faf2406af00 0x7faf2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.327+0000 7faf45e21700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faf2406af00 0x7faf2406d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ac0 tx=0x7faf30008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.331+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7faf3c053b20 con 0x7faf400cd4b0 2023-11-04T05:28:53.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.533+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faf400ae4f0 con 0x7faf400cd4b0 2023-11-04T05:28:53.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.533+0000 7faf367fc700 1 -- 172.21.15.3:0/3616992986 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7faf400ae4f0 con 0x7faf400cd4b0 2023-11-04T05:28:53.536 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:53.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faf2406af00 msgr2=0x7faf2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faf2406af00 0x7faf2406d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf30009ac0 tx=0x7faf30008040 comp rx=0 tx=0).stop 2023-11-04T05:28:53.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 msgr2=0x7faf400aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 0x7faf400aeb20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0011e0 tx=0x7faf3c00cb00 comp rx=0 tx=0).stop 2023-11-04T05:28:53.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 shutdown_connections 2023-11-04T05:28:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7faf2406af00 0x7faf2406d3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf401611d0 0x7faf400afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf4006b230 0x7faf401536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 --2- 172.21.15.3:0/3616992986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faf400cd4b0 0x7faf400aeb20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 >> 172.21.15.3:0/3616992986 conn(0x7faf40072560 msgr2=0x7faf40160020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 shutdown_connections 2023-11-04T05:28:53.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.535+0000 7faf48085700 1 -- 172.21.15.3:0/3616992986 wait complete. 2023-11-04T05:28:53.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:53.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.672+0000 7f76bbeeb700 1 Processor -- start 2023-11-04T05:28:53.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.672+0000 7f76bbeeb700 1 -- start start 2023-11-04T05:28:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76bbeeb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b406ab20 0x7f76b406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76bbeeb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76b40cdc80 0x7f76b40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76bbeeb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b40d04d0 con 0x7f76b415bb00 2023-11-04T05:28:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76bbeeb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b40cab90 con 0x7f76b40cdc80 2023-11-04T05:28:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76bbeeb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b406b440 con 0x7f76b406ab20 2023-11-04T05:28:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76ba488700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76b40cdc80 0x7f76b40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76ba488700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76b40cdc80 0x7f76b40b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52456/0 (socket says 172.21.15.3:52456) 2023-11-04T05:28:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76ba488700 1 -- 172.21.15.3:0/2249949794 learned_addr learned my addr 172.21.15.3:0/2249949794 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.673+0000 7f76b9c87700 1 --2- 172.21.15.3:0/2249949794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b406ab20 0x7f76b406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76ba488700 1 -- 172.21.15.3:0/2249949794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b406ab20 msgr2=0x7f76b406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76ba488700 1 --2- 172.21.15.3:0/2249949794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b406ab20 0x7f76b406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76ba488700 1 -- 172.21.15.3:0/2249949794 >> v1:172.21.15.188:6789/0 conn(0x7f76b415bb00 legacy=0x7f76b40b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76ba488700 1 -- 172.21.15.3:0/2249949794 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b40cada0 con 0x7f76b40cdc80 2023-11-04T05:28:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76ba488700 1 --2- 172.21.15.3:0/2249949794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b40cdc80 0x7f76b40b3390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009e50 tx=0x7f76ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76d055b0f448ac4 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76b8c85700 1 -- 172.21.15.3:0/2249949794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00e070 con 0x7f76b40cdc80 2023-11-04T05:28:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76b8c85700 1 -- 172.21.15.3:0/2249949794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac00b910 con 0x7f76b40cdc80 2023-11-04T05:28:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76b8c85700 1 -- 172.21.15.3:0/2249949794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00bcc0 con 0x7f76b40cdc80 2023-11-04T05:28:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76bbeeb700 1 -- 172.21.15.3:0/2249949794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b40cdc80 msgr2=0x7f76b40b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.674+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/2249949794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b40cdc80 0x7f76b40b3390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009e50 tx=0x7f76ac00b330 comp rx=0 tx=0).stop 2023-11-04T05:28:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.675+0000 7f76bbeeb700 1 -- 172.21.15.3:0/2249949794 shutdown_connections 2023-11-04T05:28:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.675+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/2249949794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b40cdc80 0x7f76b40b3390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.675+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/2249949794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b406ab20 0x7f76b406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.675+0000 7f76bbeeb700 1 -- 172.21.15.3:0/2249949794 >> 172.21.15.3:0/2249949794 conn(0x7f76b406df90 msgr2=0x7f76b406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.675+0000 7f76bbeeb700 1 -- 172.21.15.3:0/2249949794 shutdown_connections 2023-11-04T05:28:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.675+0000 7f76bbeeb700 1 -- 172.21.15.3:0/2249949794 wait complete. 2023-11-04T05:28:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.676+0000 7f76bbeeb700 1 Processor -- start 2023-11-04T05:28:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.676+0000 7f76bbeeb700 1 -- start start 2023-11-04T05:28:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b406ab20 0x7f76b41498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b40cdc80 0x7f76b40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 0x7f76b40b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b40b5020 con 0x7f76b415bb00 2023-11-04T05:28:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4149e20 con 0x7f76b406ab20 2023-11-04T05:28:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b40c7e70 con 0x7f76b40cdc80 2023-11-04T05:28:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 0x7f76b40b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 0x7f76b40b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52462/0 (socket says 172.21.15.3:52462) 2023-11-04T05:28:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 -- 172.21.15.3:0/597697155 learned_addr learned my addr 172.21.15.3:0/597697155 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76b9c87700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b406ab20 0x7f76b41498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76b9486700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b40cdc80 0x7f76b40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 -- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b40cdc80 msgr2=0x7f76b40c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b40cdc80 0x7f76b40c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 -- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b406ab20 msgr2=0x7f76b41498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b406ab20 0x7f76b41498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 -- 172.21.15.3:0/597697155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009b30 con 0x7f76b415bb00 2023-11-04T05:28:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76ba488700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 0x7f76b40b3bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f76ac012000 tx=0x7f76ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004800 con 0x7f76b415bb00 2023-11-04T05:28:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac004960 con 0x7f76b415bb00 2023-11-04T05:28:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac01c680 con 0x7f76b415bb00 2023-11-04T05:28:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b40b2030 con 0x7f76b415bb00 2023-11-04T05:28:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.677+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b40b25f0 con 0x7f76b415bb00 2023-11-04T05:28:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.678+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b4074df0 con 0x7f76b415bb00 2023-11-04T05:28:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.679+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76ac01c7e0 con 0x7f76b415bb00 2023-11-04T05:28:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.679+0000 7f76aa7fc700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f769806af00 0x7f769806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.679+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f76ac08b130 con 0x7f76b415bb00 2023-11-04T05:28:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.679+0000 7f76b9c87700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f769806af00 0x7f769806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.680+0000 7f76b9c87700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f769806af00 0x7f769806d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009b40 tx=0x7f76a4009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.684+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f76ac055f80 con 0x7f76b415bb00 2023-11-04T05:28:53.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.928+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76b415c240 con 0x7f76b415bb00 2023-11-04T05:28:53.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.929+0000 7f76aa7fc700 1 -- 172.21.15.3:0/597697155 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f76ac0595a0 con 0x7f76b415bb00 2023-11-04T05:28:53.931 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:53.931 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:28:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f769806af00 msgr2=0x7f769806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f769806af00 0x7f769806d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009b40 tx=0x7f76a4009210 comp rx=0 tx=0).stop 2023-11-04T05:28:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 msgr2=0x7f76b40b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 0x7f76b40b3bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f76ac012000 tx=0x7f76ac004ad0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 shutdown_connections 2023-11-04T05:28:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f769806af00 0x7f769806d3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b406ab20 0x7f76b41498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76b415bb00 0x7f76b40b3bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 --2- 172.21.15.3:0/597697155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b40cdc80 0x7f76b40c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:53.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 >> 172.21.15.3:0/597697155 conn(0x7f76b406df90 msgr2=0x7f76b40b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:53.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 shutdown_connections 2023-11-04T05:28:53.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:53.931+0000 7f76bbeeb700 1 -- 172.21.15.3:0/597697155 wait complete. 2023-11-04T05:28:53.936 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:28:53.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:54.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.068+0000 7fa5a0e7a700 1 Processor -- start 2023-11-04T05:28:54.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.068+0000 7fa5a0e7a700 1 -- start start 2023-11-04T05:28:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa5a0e7a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa59c154a90 0x7fa59c15e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa5a0e7a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa59c15e0f0 0x7fa59c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa5a0e7a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c154e70 con 0x7fa59c15eba0 2023-11-04T05:28:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa5a0e7a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c06eb20 con 0x7fa59c154a90 2023-11-04T05:28:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa5a0e7a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c06dee0 con 0x7fa59c15e0f0 2023-11-04T05:28:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa59a59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa59c154a90 0x7fa59c15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.069+0000 7fa59ad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa59c15eba0 0x7fa59c161570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57814/0 (socket says 172.21.15.3:57814) 2023-11-04T05:28:54.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59ad9d700 1 -- 172.21.15.3:0/1974794476 learned_addr learned my addr 172.21.15.3:0/1974794476 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59a59c700 1 --2- 172.21.15.3:0/1974794476 >> v2:172.21.15.3:3300/0 conn(0x7fa59c154a90 0x7fa59c15e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52466/0 (socket says 172.21.15.3:52466) 2023-11-04T05:28:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa599d9b700 1 --2- 172.21.15.3:0/1974794476 >> v2:172.21.15.188:3300/0 conn(0x7fa59c15e0f0 0x7fa59c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59a59c700 1 -- 172.21.15.3:0/1974794476 >> v2:172.21.15.188:3300/0 conn(0x7fa59c15e0f0 msgr2=0x7fa59c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59a59c700 1 --2- 172.21.15.3:0/1974794476 >> v2:172.21.15.188:3300/0 conn(0x7fa59c15e0f0 0x7fa59c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59a59c700 1 -- 172.21.15.3:0/1974794476 >> v1:172.21.15.3:6789/0 conn(0x7fa59c15eba0 legacy=0x7fa59c161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59a59c700 1 -- 172.21.15.3:0/1974794476 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c06e180 con 0x7fa59c154a90 2023-11-04T05:28:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59959a700 1 -- 172.21.15.3:0/1974794476 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946021569 0 0) 0x7fa594001490 con 0x7fa59c15eba0 2023-11-04T05:28:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59a59c700 1 --2- 172.21.15.3:0/1974794476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c154a90 0x7fa59c15e660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b2820 tx=0x7fa590009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab88fdc0af7763e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59959a700 1 -- 172.21.15.3:0/1974794476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590011830 con 0x7fa59c154a90 2023-11-04T05:28:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59959a700 1 -- 172.21.15.3:0/1974794476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa590011990 con 0x7fa59c154a90 2023-11-04T05:28:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.070+0000 7fa59959a700 1 -- 172.21.15.3:0/1974794476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590011d20 con 0x7fa59c154a90 2023-11-04T05:28:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/1974794476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c154a90 msgr2=0x7fa59c15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/1974794476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c154a90 0x7fa59c15e660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b2820 tx=0x7fa590009b20 comp rx=0 tx=0).stop 2023-11-04T05:28:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/1974794476 shutdown_connections 2023-11-04T05:28:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/1974794476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c154a90 0x7fa59c15e660 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/1974794476 >> v2:172.21.15.188:3300/0 conn(0x7fa59c15e0f0 0x7fa59c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/1974794476 >> 172.21.15.3:0/1974794476 conn(0x7fa59c071610 msgr2=0x7fa59c071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/1974794476 shutdown_connections 2023-11-04T05:28:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.071+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/1974794476 wait complete. 2023-11-04T05:28:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.072+0000 7fa5a0e7a700 1 Processor -- start 2023-11-04T05:28:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.072+0000 7fa5a0e7a700 1 -- start start 2023-11-04T05:28:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa5a0e7a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59c154a90 0x7fa59c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa5a0e7a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa59c15e0f0 0x7fa59c1466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa5a0e7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 0x7fa59c15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa5a0e7a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1556b0 con 0x7fa59c15eba0 2023-11-04T05:28:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa5a0e7a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c156010 con 0x7fa59c15e0f0 2023-11-04T05:28:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa5a0e7a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c155010 con 0x7fa59c154a90 2023-11-04T05:28:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 0x7fa59c15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 0x7fa59c15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52474/0 (socket says 172.21.15.3:52474) 2023-11-04T05:28:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 -- 172.21.15.3:0/3486117148 learned_addr learned my addr 172.21.15.3:0/3486117148 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59a59c700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59c154a90 0x7fa59c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa599d9b700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa59c15e0f0 0x7fa59c1466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 -- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59c154a90 msgr2=0x7fa59c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59c154a90 0x7fa59c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 -- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa59c15e0f0 msgr2=0x7fa59c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa59c15e0f0 0x7fa59c1466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa59ad9d700 1 -- 172.21.15.3:0/3486117148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900094a0 con 0x7fa59c15eba0 2023-11-04T05:28:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.073+0000 7fa599d9b700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa59c15e0f0 0x7fa59c1466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.074+0000 7fa59ad9d700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 0x7fa59c15a620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa59400cba0 tx=0x7fa59400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.074+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59400ab80 con 0x7fa59c15eba0 2023-11-04T05:28:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.074+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59400ad00 con 0x7fa59c15eba0 2023-11-04T05:28:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.074+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c15b000 con 0x7fa59c15eba0 2023-11-04T05:28:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.074+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594007310 con 0x7fa59c15eba0 2023-11-04T05:28:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.074+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c1552d0 con 0x7fa59c15eba0 2023-11-04T05:28:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.075+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c1557f0 con 0x7fa59c15eba0 2023-11-04T05:28:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.075+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa594007470 con 0x7fa59c15eba0 2023-11-04T05:28:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.076+0000 7fa58affd700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa57c06af00 0x7fa57c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.076+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa5940891c0 con 0x7fa59c15eba0 2023-11-04T05:28:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.076+0000 7fa59a59c700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa57c06af00 0x7fa57c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.076+0000 7fa59a59c700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa57c06af00 0x7fa57c06d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa5900090b0 tx=0x7fa590008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.080+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa594053f80 con 0x7fa59c15eba0 2023-11-04T05:28:54.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.284+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa59c154e70 con 0x7fa59c15eba0 2023-11-04T05:28:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.285+0000 7fa58affd700 1 -- 172.21.15.3:0/3486117148 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7fa59c154e70 con 0x7fa59c15eba0 2023-11-04T05:28:54.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:54.288 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:54.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.287+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa57c06af00 msgr2=0x7fa57c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.287+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa57c06af00 0x7fa57c06d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa5900090b0 tx=0x7fa590008030 comp rx=0 tx=0).stop 2023-11-04T05:28:54.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.287+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 msgr2=0x7fa59c15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.287+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 0x7fa59c15a620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa59400cba0 tx=0x7fa59400cf60 comp rx=0 tx=0).stop 2023-11-04T05:28:54.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 shutdown_connections 2023-11-04T05:28:54.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa57c06af00 0x7fa57c06d3c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c15eba0 0x7fa59c15a620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59c154a90 0x7fa59c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 --2- 172.21.15.3:0/3486117148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa59c15e0f0 0x7fa59c1466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 >> 172.21.15.3:0/3486117148 conn(0x7fa59c071610 msgr2=0x7fa59c15f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 shutdown_connections 2023-11-04T05:28:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.288+0000 7fa5a0e7a700 1 -- 172.21.15.3:0/3486117148 wait complete. 2023-11-04T05:28:54.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:28:54.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:28:54.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.428+0000 7fd737c92700 1 Processor -- start 2023-11-04T05:28:54.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.429+0000 7fd737c92700 1 -- start start 2023-11-04T05:28:54.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.429+0000 7fd737c92700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd73015eda0 0x7fd73006f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.429+0000 7fd737c92700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7300cf7e0 con 0x7fd730158360 2023-11-04T05:28:54.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.429+0000 7fd737c92700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73015e910 con 0x7fd7300668e0 2023-11-04T05:28:54.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.429+0000 7fd737c92700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300cf500 con 0x7fd73015eda0 2023-11-04T05:28:54.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd735a2e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd730158360 0x7fd7300ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57826/0 (socket says 172.21.15.3:57826) 2023-11-04T05:28:54.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd735a2e700 1 -- 172.21.15.3:0/1339179530 learned_addr learned my addr 172.21.15.3:0/1339179530 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:54.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd73522d700 1 --2- 172.21.15.3:0/1339179530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd73015eda0 0x7fd73006f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173057447 0 0) 0x7fd7300cf7e0 con 0x7fd730158360 2023-11-04T05:28:54.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd71c0035b0 con 0x7fd730158360 2023-11-04T05:28:54.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172639255 0 0) 0x7fd73015e910 con 0x7fd7300668e0 2023-11-04T05:28:54.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7300cf7e0 con 0x7fd7300668e0 2023-11-04T05:28:54.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd73522d700 1 -- 172.21.15.3:0/1339179530 >> v1:172.21.15.188:6789/0 conn(0x7fd7300668e0 legacy=0x7fd730161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd73522d700 1 -- 172.21.15.3:0/1339179530 >> v1:172.21.15.3:6789/0 conn(0x7fd730158360 legacy=0x7fd7300ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481313807 0 0) 0x7fd71c0035b0 con 0x7fd730158360 2023-11-04T05:28:54.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd73522d700 1 -- 172.21.15.3:0/1339179530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73015e2c0 con 0x7fd73015eda0 2023-11-04T05:28:54.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.430+0000 7fd73522d700 1 --2- 172.21.15.3:0/1339179530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd73015eda0 0x7fd73006f290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009870 tx=0x7fd72c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65c946176b4c8d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724001670 con 0x7fd73015eda0 2023-11-04T05:28:54.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c004030 con 0x7fd73015eda0 2023-11-04T05:28:54.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd734a2c700 1 -- 172.21.15.3:0/1339179530 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c0043c0 con 0x7fd73015eda0 2023-11-04T05:28:54.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 -- 172.21.15.3:0/1339179530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd73015eda0 msgr2=0x7fd73006f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 --2- 172.21.15.3:0/1339179530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd73015eda0 0x7fd73006f290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009870 tx=0x7fd72c009b80 comp rx=0 tx=0).stop 2023-11-04T05:28:54.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 -- 172.21.15.3:0/1339179530 shutdown_connections 2023-11-04T05:28:54.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 --2- 172.21.15.3:0/1339179530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd73015eda0 0x7fd73006f290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 -- 172.21.15.3:0/1339179530 >> 172.21.15.3:0/1339179530 conn(0x7fd7300cc860 msgr2=0x7fd7300ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:54.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 -- 172.21.15.3:0/1339179530 shutdown_connections 2023-11-04T05:28:54.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.431+0000 7fd737c92700 1 -- 172.21.15.3:0/1339179530 wait complete. 2023-11-04T05:28:54.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.432+0000 7fd737c92700 1 Processor -- start 2023-11-04T05:28:54.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.432+0000 7fd737c92700 1 -- start start 2023-11-04T05:28:54.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd737c92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7300668e0 0x7fd7300b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd737c92700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd730158360 0x7fd7301f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd737c92700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 0x7fd730155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd737c92700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73015d290 con 0x7fd73015eda0 2023-11-04T05:28:54.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd737c92700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73015e070 con 0x7fd7300668e0 2023-11-04T05:28:54.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd737c92700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73015c1f0 con 0x7fd730158360 2023-11-04T05:28:54.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd73622f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 0x7fd730155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd73622f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 0x7fd730155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52488/0 (socket says 172.21.15.3:52488) 2023-11-04T05:28:54.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd73622f700 1 -- 172.21.15.3:0/2947467016 learned_addr learned my addr 172.21.15.3:0/2947467016 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:54.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd73522d700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd730158360 0x7fd7301f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.433+0000 7fd735a2e700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7300668e0 0x7fd7300b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73622f700 1 -- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd730158360 msgr2=0x7fd7301f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73622f700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd730158360 0x7fd7301f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73622f700 1 -- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7300668e0 msgr2=0x7fd7300b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73622f700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7300668e0 0x7fd7300b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73622f700 1 -- 172.21.15.3:0/2947467016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0094a0 con 0x7fd73015eda0 2023-11-04T05:28:54.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73522d700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd730158360 0x7fd7301f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:54.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd735a2e700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7300668e0 0x7fd7300b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:54.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd73622f700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 0x7fd730155490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd72800cc20 tx=0x7fd72800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728003c00 con 0x7fd73015eda0 2023-11-04T05:28:54.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728003d60 con 0x7fd73015eda0 2023-11-04T05:28:54.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73015c4d0 con 0x7fd73015eda0 2023-11-04T05:28:54.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800aae0 con 0x7fd73015eda0 2023-11-04T05:28:54.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.434+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73015cd70 con 0x7fd73015eda0 2023-11-04T05:28:54.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.435+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7300730c0 con 0x7fd73015eda0 2023-11-04T05:28:54.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.435+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd72800ac40 con 0x7fd73015eda0 2023-11-04T05:28:54.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.436+0000 7fd7227fc700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd71406af00 0x7fd71406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.436+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd7280898f0 con 0x7fd73015eda0 2023-11-04T05:28:54.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.436+0000 7fd735a2e700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd71406af00 0x7fd71406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.437+0000 7fd735a2e700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd71406af00 0x7fd71406d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7240017a0 tx=0x7fd724000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.440+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd7280546c0 con 0x7fd73015eda0 2023-11-04T05:28:54.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.642+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7300cc0c0 con 0x7fd73015eda0 2023-11-04T05:28:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.642+0000 7fd7227fc700 1 -- 172.21.15.3:0/2947467016 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd728057ce0 con 0x7fd73015eda0 2023-11-04T05:28:54.645 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:28:54.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.644+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd71406af00 msgr2=0x7fd71406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.644+0000 7fd737c92700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd71406af00 0x7fd71406d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7240017a0 tx=0x7fd724000bc0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.644+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 msgr2=0x7fd730155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.644+0000 7fd737c92700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 0x7fd730155490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd72800cc20 tx=0x7fd72800cf30 comp rx=0 tx=0).stop 2023-11-04T05:28:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.644+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 shutdown_connections 2023-11-04T05:28:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd71406af00 0x7fd71406d3c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7300668e0 0x7fd7300b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73015eda0 0x7fd730155490 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 --2- 172.21.15.3:0/2947467016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd730158360 0x7fd7301f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 >> 172.21.15.3:0/2947467016 conn(0x7fd7300cc860 msgr2=0x7fd73015f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 shutdown_connections 2023-11-04T05:28:54.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.645+0000 7fd737c92700 1 -- 172.21.15.3:0/2947467016 wait complete. 2023-11-04T05:28:54.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:28:54.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.784+0000 7fbd5fa26700 1 Processor -- start 2023-11-04T05:28:54.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.785+0000 7fbd5fa26700 1 -- start start 2023-11-04T05:28:54.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.785+0000 7fbd5fa26700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd5815ba10 0x7fbd580c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.785+0000 7fbd5fa26700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580d0d70 0x7fbd580b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.785+0000 7fbd5fa26700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd58070a90 con 0x7fbd580d11f0 2023-11-04T05:28:54.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.785+0000 7fbd5fa26700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd580718f0 con 0x7fbd5815ba10 2023-11-04T05:28:54.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.785+0000 7fbd5fa26700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580c5720 con 0x7fbd580d0d70 2023-11-04T05:28:54.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5d7c2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd5815ba10 0x7fbd580c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5dfc3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbd580d11f0 0x7fbd5815f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57844/0 (socket says 172.21.15.3:57844) 2023-11-04T05:28:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5dfc3700 1 -- 172.21.15.3:0/2080666093 learned_addr learned my addr 172.21.15.3:0/2080666093 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5cfc1700 1 --2- 172.21.15.3:0/2080666093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580d0d70 0x7fbd580b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/2080666093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580d0d70 msgr2=0x7fbd580b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5d7c2700 1 --2- 172.21.15.3:0/2080666093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580d0d70 0x7fbd580b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/2080666093 >> v1:172.21.15.3:6789/0 conn(0x7fbd580d11f0 legacy=0x7fbd5815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/2080666093 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58070ce0 con 0x7fbd5815ba10 2023-11-04T05:28:54.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd5d7c2700 1 --2- 172.21.15.3:0/2080666093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd5815ba10 0x7fbd580c51e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009840 tx=0x7fbd48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a24fa3f5ac02782 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd4ffff700 1 -- 172.21.15.3:0/2080666093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011830 con 0x7fbd5815ba10 2023-11-04T05:28:54.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd4ffff700 1 -- 172.21.15.3:0/2080666093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48011990 con 0x7fbd5815ba10 2023-11-04T05:28:54.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.786+0000 7fbd4ffff700 1 -- 172.21.15.3:0/2080666093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011d20 con 0x7fbd5815ba10 2023-11-04T05:28:54.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 -- 172.21.15.3:0/2080666093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd5815ba10 msgr2=0x7fbd580c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/2080666093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd5815ba10 0x7fbd580c51e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009840 tx=0x7fbd48009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:54.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 -- 172.21.15.3:0/2080666093 shutdown_connections 2023-11-04T05:28:54.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/2080666093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd5815ba10 0x7fbd580c51e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/2080666093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580d0d70 0x7fbd580b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 -- 172.21.15.3:0/2080666093 >> 172.21.15.3:0/2080666093 conn(0x7fbd580ce110 msgr2=0x7fbd580ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:54.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 -- 172.21.15.3:0/2080666093 shutdown_connections 2023-11-04T05:28:54.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.787+0000 7fbd5fa26700 1 -- 172.21.15.3:0/2080666093 wait complete. 2023-11-04T05:28:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.788+0000 7fbd5fa26700 1 Processor -- start 2023-11-04T05:28:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.789+0000 7fbd5fa26700 1 -- start start 2023-11-04T05:28:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.789+0000 7fbd5fa26700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 0x7fbd5814b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.789+0000 7fbd5fa26700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5815ba10 0x7fbd580c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.789+0000 7fbd5fa26700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580c6f70 0x7fbd5815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5fa26700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580c7a60 con 0x7fbd580d0d70 2023-11-04T05:28:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5fa26700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580c74a0 con 0x7fbd5815ba10 2023-11-04T05:28:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5fa26700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580c7780 con 0x7fbd580c6f70 2023-11-04T05:28:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 0x7fbd5814b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 0x7fbd5814b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52498/0 (socket says 172.21.15.3:52498) 2023-11-04T05:28:54.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/627728183 learned_addr learned my addr 172.21.15.3:0/627728183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:54.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5cfc1700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5815ba10 0x7fbd580c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5dfc3700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580c6f70 0x7fbd5815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580c6f70 msgr2=0x7fbd5815c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580c6f70 0x7fbd5815c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5815ba10 msgr2=0x7fbd580c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5815ba10 0x7fbd580c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 -- 172.21.15.3:0/627728183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480094a0 con 0x7fbd580d0d70 2023-11-04T05:28:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5cfc1700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5815ba10 0x7fbd580c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5d7c2700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 0x7fbd5814b9a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480036a0 con 0x7fbd580d0d70 2023-11-04T05:28:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4801e070 con 0x7fbd580d0d70 2023-11-04T05:28:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd580b5520 con 0x7fbd580d0d70 2023-11-04T05:28:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48003a60 con 0x7fbd580d0d70 2023-11-04T05:28:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.790+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5814bee0 con 0x7fbd580d0d70 2023-11-04T05:28:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.791+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd580b5380 con 0x7fbd580d0d70 2023-11-04T05:28:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.792+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd48003bc0 con 0x7fbd580d0d70 2023-11-04T05:28:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.792+0000 7fbd4dffb700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbd3c06af00 0x7fbd3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.792+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd4808a950 con 0x7fbd580d0d70 2023-11-04T05:28:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.793+0000 7fbd5cfc1700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbd3c06af00 0x7fbd3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.793+0000 7fbd5cfc1700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbd3c06af00 0x7fbd3c06d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd580d0600 tx=0x7fbd54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.797+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fbd580b5380 con 0x7fbd580d0d70 2023-11-04T05:28:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.999+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd580b5380 con 0x7fbd580d0d70 2023-11-04T05:28:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:54.999+0000 7fbd4dffb700 1 -- 172.21.15.3:0/627728183 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd48020020 con 0x7fbd580d0d70 2023-11-04T05:28:55.002 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:55.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbd3c06af00 msgr2=0x7fbd3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbd3c06af00 0x7fbd3c06d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd580d0600 tx=0x7fbd54008040 comp rx=0 tx=0).stop 2023-11-04T05:28:55.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 msgr2=0x7fbd5814b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 0x7fbd5814b9a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48004750 comp rx=0 tx=0).stop 2023-11-04T05:28:55.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 shutdown_connections 2023-11-04T05:28:55.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbd3c06af00 0x7fbd3c06d3c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd580c6f70 0x7fbd5815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd580d0d70 0x7fbd5814b9a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 --2- 172.21.15.3:0/627728183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5815ba10 0x7fbd580c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 >> 172.21.15.3:0/627728183 conn(0x7fbd580ce110 msgr2=0x7fbd5815aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 shutdown_connections 2023-11-04T05:28:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.001+0000 7fbd5fa26700 1 -- 172.21.15.3:0/627728183 wait complete. 2023-11-04T05:28:55.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:28:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.142+0000 7f351445e700 1 Processor -- start 2023-11-04T05:28:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.142+0000 7f351445e700 1 -- start start 2023-11-04T05:28:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f351445e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f350c15dbc0 0x7f350c0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f351445e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d2510 0x7f350c06bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f351445e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0ce330 con 0x7f350c0d1600 2023-11-04T05:28:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f351445e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c0d1b70 con 0x7f350c15dbc0 2023-11-04T05:28:55.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f351445e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0ce560 con 0x7f350c0d2510 2023-11-04T05:28:55.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f350c15dbc0 0x7f350c0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f350c15dbc0 0x7f350c0cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52506/0 (socket says 172.21.15.3:52506) 2023-11-04T05:28:55.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 -- 172.21.15.3:0/1092949043 learned_addr learned my addr 172.21.15.3:0/1092949043 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:55.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35129fb700 1 --2- 172.21.15.3:0/1092949043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d2510 0x7f350c06bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 -- 172.21.15.3:0/1092949043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d2510 msgr2=0x7f350c06bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:55.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 --2- 172.21.15.3:0/1092949043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d2510 0x7f350c06bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 -- 172.21.15.3:0/1092949043 >> v1:172.21.15.188:6789/0 conn(0x7f350c0d1600 legacy=0x7f350c0d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.143+0000 7f35121fa700 1 -- 172.21.15.3:0/1092949043 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0ce840 con 0x7f350c15dbc0 2023-11-04T05:28:55.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f35121fa700 1 --2- 172.21.15.3:0/1092949043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c0cf320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34fc006fd0 tx=0x7f34fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a668aa59a96fc241 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f35111f8700 1 -- 172.21.15.3:0/1092949043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc005e30 con 0x7f350c15dbc0 2023-11-04T05:28:55.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f35111f8700 1 -- 172.21.15.3:0/1092949043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc004030 con 0x7f350c15dbc0 2023-11-04T05:28:55.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f35111f8700 1 -- 172.21.15.3:0/1092949043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0043e0 con 0x7f350c15dbc0 2023-11-04T05:28:55.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 -- 172.21.15.3:0/1092949043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 msgr2=0x7f350c0cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 --2- 172.21.15.3:0/1092949043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c0cf320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34fc006fd0 tx=0x7f34fc009bd0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 -- 172.21.15.3:0/1092949043 shutdown_connections 2023-11-04T05:28:55.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 --2- 172.21.15.3:0/1092949043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d2510 0x7f350c06bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 --2- 172.21.15.3:0/1092949043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c0cf320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 -- 172.21.15.3:0/1092949043 >> 172.21.15.3:0/1092949043 conn(0x7f350c09cb40 msgr2=0x7f350c0d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:55.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 -- 172.21.15.3:0/1092949043 shutdown_connections 2023-11-04T05:28:55.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.144+0000 7f351445e700 1 -- 172.21.15.3:0/1092949043 wait complete. 2023-11-04T05:28:55.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 Processor -- start 2023-11-04T05:28:55.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 -- start start 2023-11-04T05:28:55.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d1600 0x7f350c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d2510 0x7f350c15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c09f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0cdc10 con 0x7f350c15dbc0 2023-11-04T05:28:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0cb3a0 con 0x7f350c0d2510 2023-11-04T05:28:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.146+0000 7f351445e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0cb610 con 0x7f350c0d1600 2023-11-04T05:28:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c09f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c09f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52516/0 (socket says 172.21.15.3:52516) 2023-11-04T05:28:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 -- 172.21.15.3:0/526426181 learned_addr learned my addr 172.21.15.3:0/526426181 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35121fa700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d1600 0x7f350c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35119f9700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d2510 0x7f350c15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 -- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d1600 msgr2=0x7f350c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d1600 0x7f350c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 -- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d2510 msgr2=0x7f350c15c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d2510 0x7f350c15c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 -- 172.21.15.3:0/526426181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc009500 con 0x7f350c15dbc0 2023-11-04T05:28:55.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35119f9700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d2510 0x7f350c15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:55.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f35129fb700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c09f840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3504007de0 tx=0x7f350400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400ede0 con 0x7f350c15dbc0 2023-11-04T05:28:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3504014420 con 0x7f350c15dbc0 2023-11-04T05:28:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c0cb8f0 con 0x7f350c15dbc0 2023-11-04T05:28:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040147d0 con 0x7f350c15dbc0 2023-11-04T05:28:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.147+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c15c710 con 0x7f350c15dbc0 2023-11-04T05:28:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.148+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c074df0 con 0x7f350c15dbc0 2023-11-04T05:28:55.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.149+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3504014930 con 0x7f350c15dbc0 2023-11-04T05:28:55.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.149+0000 7f3502ffd700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f34f006af00 0x7f34f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.149+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f350408a070 con 0x7f350c15dbc0 2023-11-04T05:28:55.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.149+0000 7f35121fa700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f34f006af00 0x7f34f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.150+0000 7f35121fa700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f34f006af00 0x7f34f006d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34fc000c00 tx=0x7f34fc008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.154+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f3504054e30 con 0x7f350c15dbc0 2023-11-04T05:28:55.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.354+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f350c0b3440 con 0x7f350c15dbc0 2023-11-04T05:28:55.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.354+0000 7f3502ffd700 1 -- 172.21.15.3:0/526426181 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3504058450 con 0x7f350c15dbc0 2023-11-04T05:28:55.357 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:28:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f34f006af00 msgr2=0x7f34f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f34f006af00 0x7f34f006d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34fc000c00 tx=0x7f34fc008070 comp rx=0 tx=0).stop 2023-11-04T05:28:55.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 msgr2=0x7f350c09f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c09f840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3504007de0 tx=0x7f350400e3c0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 shutdown_connections 2023-11-04T05:28:55.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f34f006af00 0x7f34f006d3c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f350c15dbc0 0x7f350c09f840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c0d1600 0x7f350c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 --2- 172.21.15.3:0/526426181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d2510 0x7f350c15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 >> 172.21.15.3:0/526426181 conn(0x7f350c09cb40 msgr2=0x7f350c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 shutdown_connections 2023-11-04T05:28:55.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.357+0000 7f351445e700 1 -- 172.21.15.3:0/526426181 wait complete. 2023-11-04T05:28:55.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:28:55.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.497+0000 7fbbf5c5d700 1 Processor -- start 2023-11-04T05:28:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.497+0000 7fbbf5c5d700 1 -- start start 2023-11-04T05:28:55.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.498+0000 7fbbf5c5d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbf0168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.498+0000 7fbbf5c5d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00ceed0 con 0x7fbbf00b2620 2023-11-04T05:28:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.498+0000 7fbbf5c5d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00d1520 con 0x7fbbf015d9a0 2023-11-04T05:28:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.498+0000 7fbbf5c5d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00d46d0 con 0x7fbbf00d0800 2023-11-04T05:28:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbeeffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbbf00b2620 0x7fbbf00d00f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57870/0 (socket says 172.21.15.3:57870) 2023-11-04T05:28:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbeeffd700 1 -- 172.21.15.3:0/1696363792 learned_addr learned my addr 172.21.15.3:0/1696363792 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbeffff700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbf0168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496190585 0 0) 0x7fbbf00ceed0 con 0x7fbbf00b2620 2023-11-04T05:28:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbdc0035b0 con 0x7fbbf00b2620 2023-11-04T05:28:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3824635675 0 0) 0x7fbbdc0035b0 con 0x7fbbf00b2620 2023-11-04T05:28:55.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 msgr2=0x7fbbf0168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbf0168950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 >> v1:172.21.15.188:6789/0 conn(0x7fbbf015d9a0 legacy=0x7fbbf00d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00cf120 con 0x7fbbf00b2620 2023-11-04T05:28:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fbbd8001670 con 0x7fbbf00b2620 2023-11-04T05:28:55.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 >> v1:172.21.15.3:6789/0 conn(0x7fbbf00b2620 legacy=0x7fbbf00d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc003a30 0x7fbbdc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbdc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbdc004d90 0x7fbbdc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00ba10 con 0x7fbbdc004d90 2023-11-04T05:28:55.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00bb80 con 0x7fbbdc003a30 2023-11-04T05:28:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00bd40 con 0x7fbbf00d0800 2023-11-04T05:28:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbeffff700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbdc004d90 0x7fbbdc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbeeffd700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbdc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.499+0000 7fbbef7fe700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc003a30 0x7fbbdc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 -- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 msgr2=0x7fbbdc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbdc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 -- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc003a30 msgr2=0x7fbbdc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc003a30 0x7fbbdc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 -- 172.21.15.3:0/1696363792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0035b0 con 0x7fbbdc004d90 2023-11-04T05:28:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeeffd700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf00d0800 0x7fbbdc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:55.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbef7fe700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc003a30 0x7fbbdc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbdc004d90 0x7fbbdc005210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009fc0 tx=0x7fbbdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fa5db8e0578e6f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800ec30 con 0x7fbbdc004d90 2023-11-04T05:28:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe800f040 con 0x7fbbdc004d90 2023-11-04T05:28:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbee7fc700 1 -- 172.21.15.3:0/1696363792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8014480 con 0x7fbbdc004d90 2023-11-04T05:28:55.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/1696363792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbdc004d90 msgr2=0x7fbbdc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/1696363792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbdc004d90 0x7fbbdc005210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009fc0 tx=0x7fbbdc005880 comp rx=0 tx=0).stop 2023-11-04T05:28:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbeffff700 1 -- 172.21.15.3:0/1696363792 reap_dead start 2023-11-04T05:28:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/1696363792 shutdown_connections 2023-11-04T05:28:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/1696363792 >> 172.21.15.3:0/1696363792 conn(0x7fbbf008aff0 msgr2=0x7fbbf00d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:55.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.500+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/1696363792 shutdown_connections 2023-11-04T05:28:55.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.501+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/1696363792 wait complete. 2023-11-04T05:28:55.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.502+0000 7fbbf5c5d700 1 Processor -- start 2023-11-04T05:28:55.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.502+0000 7fbbf5c5d700 1 -- start start 2023-11-04T05:28:55.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc004d90 0x7fbbf015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 0x7fbbf006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf01f3790 0x7fbbf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00cce20 con 0x7fbbf01f3310 2023-11-04T05:28:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00c8740 con 0x7fbbf01f3790 2023-11-04T05:28:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00cc1a0 con 0x7fbbdc004d90 2023-11-04T05:28:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 0x7fbbf006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 0x7fbbf006cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52534/0 (socket says 172.21.15.3:52534) 2023-11-04T05:28:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 -- 172.21.15.3:0/2748926164 learned_addr learned my addr 172.21.15.3:0/2748926164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:55.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbef7fe700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc004d90 0x7fbbf015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeffff700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf01f3790 0x7fbbf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 -- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc004d90 msgr2=0x7fbbf015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc004d90 0x7fbbf015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 -- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf01f3790 msgr2=0x7fbbf006d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf01f3790 0x7fbbf006d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 -- 172.21.15.3:0/2748926164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00cc480 con 0x7fbbf01f3310 2023-11-04T05:28:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbeeffd700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 0x7fbbf006cdc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e920 tx=0x7fbbd800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800eec0 con 0x7fbbf01f3310 2023-11-04T05:28:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800c7e0 con 0x7fbbf01f3310 2023-11-04T05:28:55.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf00cc760 con 0x7fbbf01f3310 2023-11-04T05:28:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800cb50 con 0x7fbbf01f3310 2023-11-04T05:28:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.503+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf00c9c20 con 0x7fbbf01f3310 2023-11-04T05:28:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.504+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf00d0e60 con 0x7fbbf01f3310 2023-11-04T05:28:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.505+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbbd800ccb0 con 0x7fbbf01f3310 2023-11-04T05:28:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.505+0000 7fbbe7fff700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbbd006af00 0x7fbbd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.505+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbbd80897c0 con 0x7fbbf01f3310 2023-11-04T05:28:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.505+0000 7fbbef7fe700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbbd006af00 0x7fbbd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.506+0000 7fbbef7fe700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbbd006af00 0x7fbbd006d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0007bd0 tx=0x7fbbe0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.510+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fbbd8054580 con 0x7fbbf01f3310 2023-11-04T05:28:55.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.712+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbbf015be30 con 0x7fbbf01f3310 2023-11-04T05:28:55.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.713+0000 7fbbe7fff700 1 -- 172.21.15.3:0/2748926164 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbbd8057ba0 con 0x7fbbf01f3310 2023-11-04T05:28:55.715 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:28:55.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbbd006af00 msgr2=0x7fbbd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbbd006af00 0x7fbbd006d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0007bd0 tx=0x7fbbe0009040 comp rx=0 tx=0).stop 2023-11-04T05:28:55.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 msgr2=0x7fbbf006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 0x7fbbf006cdc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e920 tx=0x7fbbd800ece0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 shutdown_connections 2023-11-04T05:28:55.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbbd006af00 0x7fbbd006d3c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf01f3790 0x7fbbf006d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc004d90 0x7fbbf015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 --2- 172.21.15.3:0/2748926164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf01f3310 0x7fbbf006cdc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 >> 172.21.15.3:0/2748926164 conn(0x7fbbf008aff0 msgr2=0x7fbbf00b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:55.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 shutdown_connections 2023-11-04T05:28:55.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.715+0000 7fbbf5c5d700 1 -- 172.21.15.3:0/2748926164 wait complete. 2023-11-04T05:28:55.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:55.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.852+0000 7f8c59da6700 1 Processor -- start 2023-11-04T05:28:55.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.852+0000 7f8c59da6700 1 -- start start 2023-11-04T05:28:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c59da6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c540cf400 0x7f8c540b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c59da6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540b3140 con 0x7f8c5415a7e0 2023-11-04T05:28:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c59da6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540cbc70 con 0x7f8c540cb4e0 2023-11-04T05:28:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c59da6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c540cbf50 con 0x7f8c540cf400 2023-11-04T05:28:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c53fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c540cf400 0x7f8c540b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c537fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8c5415a7e0 0x7f8c540ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57888/0 (socket says 172.21.15.3:57888) 2023-11-04T05:28:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.853+0000 7f8c537fe700 1 -- 172.21.15.3:0/2450677099 learned_addr learned my addr 172.21.15.3:0/2450677099 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c53fff700 1 -- 172.21.15.3:0/2450677099 >> v1:172.21.15.188:6789/0 conn(0x7f8c540cb4e0 legacy=0x7f8c540cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c53fff700 1 -- 172.21.15.3:0/2450677099 >> v1:172.21.15.3:6789/0 conn(0x7f8c5415a7e0 legacy=0x7f8c540ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c527fc700 1 -- 172.21.15.3:0/2450677099 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181787338 0 0) 0x7f8c540b3140 con 0x7f8c5415a7e0 2023-11-04T05:28:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c53fff700 1 -- 172.21.15.3:0/2450677099 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540cc230 con 0x7f8c540cf400 2023-11-04T05:28:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c53fff700 1 --2- 172.21.15.3:0/2450677099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c540cf400 0x7f8c540b2c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009840 tx=0x7f8c4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db2ea04a37aa1c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c527fc700 1 -- 172.21.15.3:0/2450677099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004230 con 0x7f8c540cf400 2023-11-04T05:28:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c527fc700 1 -- 172.21.15.3:0/2450677099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c004390 con 0x7f8c540cf400 2023-11-04T05:28:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.854+0000 7f8c527fc700 1 -- 172.21.15.3:0/2450677099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0036a0 con 0x7f8c540cf400 2023-11-04T05:28:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 -- 172.21.15.3:0/2450677099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c540cf400 msgr2=0x7f8c540b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 --2- 172.21.15.3:0/2450677099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c540cf400 0x7f8c540b2c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009840 tx=0x7f8c4c009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 -- 172.21.15.3:0/2450677099 shutdown_connections 2023-11-04T05:28:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 --2- 172.21.15.3:0/2450677099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c540cf400 0x7f8c540b2c00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 -- 172.21.15.3:0/2450677099 >> 172.21.15.3:0/2450677099 conn(0x7f8c540cfd50 msgr2=0x7f8c540d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:55.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 -- 172.21.15.3:0/2450677099 shutdown_connections 2023-11-04T05:28:55.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.855+0000 7f8c59da6700 1 -- 172.21.15.3:0/2450677099 wait complete. 2023-11-04T05:28:55.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.856+0000 7f8c59da6700 1 Processor -- start 2023-11-04T05:28:55.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.856+0000 7f8c59da6700 1 -- start start 2023-11-04T05:28:55.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c59da6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c540cb4e0 0x7f8c5414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c59da6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c540cf400 0x7f8c540b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c59da6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 0x7f8c540b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c59da6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b2bb0 con 0x7f8c5415a7e0 2023-11-04T05:28:55.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c59da6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5415d4a0 con 0x7f8c540cb4e0 2023-11-04T05:28:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c59da6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5415c920 con 0x7f8c540cf400 2023-11-04T05:28:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 0x7f8c540b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 0x7f8c540b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52550/0 (socket says 172.21.15.3:52550) 2023-11-04T05:28:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 -- 172.21.15.3:0/1019354015 learned_addr learned my addr 172.21.15.3:0/1019354015 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c537fe700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c540cb4e0 0x7f8c5414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 -- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c540cf400 msgr2=0x7f8c540b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c540cf400 0x7f8c540b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 -- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c540cb4e0 msgr2=0x7f8c5414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c540cb4e0 0x7f8c5414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:55.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 -- 172.21.15.3:0/1019354015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0094a0 con 0x7f8c5415a7e0 2023-11-04T05:28:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c537fe700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c540cb4e0 0x7f8c5414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:28:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.857+0000 7f8c53fff700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 0x7f8c540b5360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009f50 tx=0x7f8c4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.858+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c011630 con 0x7f8c5415a7e0 2023-11-04T05:28:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.858+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c01e070 con 0x7f8c5415a7e0 2023-11-04T05:28:55.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.858+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0119d0 con 0x7f8c5415a7e0 2023-11-04T05:28:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.858+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5415cc00 con 0x7f8c5415a7e0 2023-11-04T05:28:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.858+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5415d1c0 con 0x7f8c5415a7e0 2023-11-04T05:28:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.859+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540b2d20 con 0x7f8c5415a7e0 2023-11-04T05:28:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.859+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c4c011b30 con 0x7f8c5415a7e0 2023-11-04T05:28:55.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.860+0000 7f8c4bfff700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c3406af00 0x7f8c3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:55.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.860+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8c4c08abf0 con 0x7f8c5415a7e0 2023-11-04T05:28:55.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.860+0000 7f8c537fe700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c3406af00 0x7f8c3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:55.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.860+0000 7f8c537fe700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c3406af00 0x7f8c3406d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c440012e0 tx=0x7f8c44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:55.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:55.864+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8c4c055a40 con 0x7f8c5415a7e0 2023-11-04T05:28:56.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.114+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c54074df0 con 0x7f8c5415a7e0 2023-11-04T05:28:56.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.114+0000 7f8c4bfff700 1 -- 172.21.15.3:0/1019354015 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f8c4c020070 con 0x7f8c5415a7e0 2023-11-04T05:28:56.117 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:56.117 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:28:56.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c3406af00 msgr2=0x7f8c3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c3406af00 0x7f8c3406d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c440012e0 tx=0x7f8c44009040 comp rx=0 tx=0).stop 2023-11-04T05:28:56.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 msgr2=0x7f8c540b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 0x7f8c540b5360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009f50 tx=0x7f8c4c004750 comp rx=0 tx=0).stop 2023-11-04T05:28:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 shutdown_connections 2023-11-04T05:28:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c3406af00 0x7f8c3406d3c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c540cb4e0 0x7f8c5414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c5415a7e0 0x7f8c540b5360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 --2- 172.21.15.3:0/1019354015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c540cf400 0x7f8c540b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 >> 172.21.15.3:0/1019354015 conn(0x7f8c540cfd50 msgr2=0x7f8c541574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 shutdown_connections 2023-11-04T05:28:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.117+0000 7f8c59da6700 1 -- 172.21.15.3:0/1019354015 wait complete. 2023-11-04T05:28:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:28:56.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:28:56.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.254+0000 7fa4f38c8700 1 Processor -- start 2023-11-04T05:28:56.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.254+0000 7fa4f38c8700 1 -- start start 2023-11-04T05:28:56.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.255+0000 7fa4f38c8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4ec0728b0 0x7fa4ec1688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.255+0000 7fa4f38c8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec0cda00 con 0x7fa4ec06ab20 2023-11-04T05:28:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.255+0000 7fa4f38c8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec0cdce0 con 0x7fa4ec0cc7a0 2023-11-04T05:28:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.255+0000 7fa4f38c8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cf050 con 0x7fa4ec0728b0 2023-11-04T05:28:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.255+0000 7fa4f1664700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa4ec06ab20 0x7fa4ec0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57902/0 (socket says 172.21.15.3:57902) 2023-11-04T05:28:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.255+0000 7fa4f1664700 1 -- 172.21.15.3:0/4196945010 learned_addr learned my addr 172.21.15.3:0/4196945010 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:28:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581027725 0 0) 0x7fa4ec0cda00 con 0x7fa4ec06ab20 2023-11-04T05:28:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1e65700 1 --2- 172.21.15.3:0/4196945010 >> v2:172.21.15.188:3300/0 conn(0x7fa4ec0728b0 0x7fa4ec1688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d80035b0 con 0x7fa4ec06ab20 2023-11-04T05:28:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 699886525 0 0) 0x7fa4d80035b0 con 0x7fa4ec06ab20 2023-11-04T05:28:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 >> v2:172.21.15.188:3300/0 conn(0x7fa4ec0728b0 msgr2=0x7fa4ec1688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:56.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 --2- 172.21.15.3:0/4196945010 >> v2:172.21.15.188:3300/0 conn(0x7fa4ec0728b0 0x7fa4ec1688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 >> v1:172.21.15.188:6789/0 conn(0x7fa4ec0cc7a0 legacy=0x7fa4ec06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec0720b0 con 0x7fa4ec06ab20 2023-11-04T05:28:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fa4dc001670 con 0x7fa4ec06ab20 2023-11-04T05:28:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 >> v1:172.21.15.3:6789/0 conn(0x7fa4ec06ab20 legacy=0x7fa4ec0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d8003a00 0x7fa4d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8004850 0x7fa4d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d8005210 0x7fa4d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cf050 con 0x7fa4d8003a00 2023-11-04T05:28:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d800c1b0 con 0x7fa4d8005210 2023-11-04T05:28:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d800c370 con 0x7fa4d8004850 2023-11-04T05:28:56.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1664700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d8003a00 0x7fa4d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1e65700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d8005210 0x7fa4d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f0e63700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8004850 0x7fa4d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1664700 1 -- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8004850 msgr2=0x7fa4d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:56.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1664700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8004850 0x7fa4d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1664700 1 -- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d8005210 msgr2=0x7fa4d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:56.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1664700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d8005210 0x7fa4d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1664700 1 -- 172.21.15.3:0/4196945010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d80035b0 con 0x7fa4d8003a00 2023-11-04T05:28:56.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1e65700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d8005210 0x7fa4d800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:56.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.256+0000 7fa4f1e65700 1 -- 172.21.15.3:0/4196945010 reap_dead start 2023-11-04T05:28:56.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f1664700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d8003a00 0x7fa4d8003e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00c240 tx=0x7fa4d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf78fb94e1f4dbd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc007c10 con 0x7fa4d8003a00 2023-11-04T05:28:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00cde0 con 0x7fa4d8003a00 2023-11-04T05:28:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4e3fff700 1 -- 172.21.15.3:0/4196945010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc005ba0 con 0x7fa4d8003a00 2023-11-04T05:28:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 -- 172.21.15.3:0/4196945010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d8003a00 msgr2=0x7fa4d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d8003a00 0x7fa4d8003e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00c240 tx=0x7fa4d8005880 comp rx=0 tx=0).stop 2023-11-04T05:28:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 -- 172.21.15.3:0/4196945010 shutdown_connections 2023-11-04T05:28:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/4196945010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d8003a00 0x7fa4d8003e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 -- 172.21.15.3:0/4196945010 >> 172.21.15.3:0/4196945010 conn(0x7fa4ec06df90 msgr2=0x7fa4ec06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 -- 172.21.15.3:0/4196945010 shutdown_connections 2023-11-04T05:28:56.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.257+0000 7fa4f38c8700 1 -- 172.21.15.3:0/4196945010 wait complete. 2023-11-04T05:28:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 Processor -- start 2023-11-04T05:28:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 -- start start 2023-11-04T05:28:56.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8003a00 0x7fa4ec0c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 0x7fa4ec1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4ec06ea60 0x7fa4ec1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1f7810 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0c7120 con 0x7fa4ec06ea60 2023-11-04T05:28:56.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.259+0000 7fa4f38c8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cb5f0 con 0x7fa4d8003a00 2023-11-04T05:28:56.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 0x7fa4ec1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f1664700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8003a00 0x7fa4ec0c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 0x7fa4ec1f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52568/0 (socket says 172.21.15.3:52568) 2023-11-04T05:28:56.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f1664700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8003a00 0x7fa4ec0c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43622/0 (socket says 172.21.15.3:43622) 2023-11-04T05:28:56.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 -- 172.21.15.3:0/2537911468 learned_addr learned my addr 172.21.15.3:0/2537911468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:56.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f1e65700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4ec06ea60 0x7fa4ec1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 -- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8003a00 msgr2=0x7fa4ec0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8003a00 0x7fa4ec0c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 -- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4ec06ea60 msgr2=0x7fa4ec1f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4ec06ea60 0x7fa4ec1f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 -- 172.21.15.3:0/2537911468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec1f7530 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f0e63700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 0x7fa4ec1f6950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800e550 tx=0x7fa4e800e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:56.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80052e0 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8005440 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ec0c6100 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80038f0 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.260+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec0c66c0 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.261+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ec0c6880 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.262+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4e8003a50 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.262+0000 7fa4e1ffb700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa4d006af00 0x7fa4d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.262+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa4e8088e80 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.262+0000 7fa4f1664700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa4d006af00 0x7fa4d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.263+0000 7fa4f1664700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa4d006af00 0x7fa4d006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:56.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.267+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa4e8053ce0 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.469+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4ec074df0 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.470+0000 7fa4e1ffb700 1 -- 172.21.15.3:0/2537911468 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18286 (secure 0 0 0) 0x7fa4e8019020 con 0x7fa4ec06e5e0 2023-11-04T05:28:56.473 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:56.473 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:28:26.264980+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:28:56.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa4d006af00 msgr2=0x7fa4d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa4d006af00 0x7fa4d006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc007000 comp rx=0 tx=0).stop 2023-11-04T05:28:56.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 msgr2=0x7fa4ec1f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 0x7fa4ec1f6950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800e550 tx=0x7fa4e800e860 comp rx=0 tx=0).stop 2023-11-04T05:28:56.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 shutdown_connections 2023-11-04T05:28:56.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa4d006af00 0x7fa4d006d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4ec06ea60 0x7fa4ec1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d8003a00 0x7fa4ec0c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 --2- 172.21.15.3:0/2537911468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec06e5e0 0x7fa4ec1f6950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 >> 172.21.15.3:0/2537911468 conn(0x7fa4ec06df90 msgr2=0x7fa4ec0cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:56.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 shutdown_connections 2023-11-04T05:28:56.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.472+0000 7fa4f38c8700 1 -- 172.21.15.3:0/2537911468 wait complete. 2023-11-04T05:28:56.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:28:56.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.609+0000 7fd55d1f4700 1 Processor -- start 2023-11-04T05:28:56.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.609+0000 7fd55d1f4700 1 -- start start 2023-11-04T05:28:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.610+0000 7fd55d1f4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 0x7fd5580cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.610+0000 7fd55d1f4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd55815bd70 0x7fd5580ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.610+0000 7fd55d1f4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5580b75f0 con 0x7fd5581526c0 2023-11-04T05:28:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.610+0000 7fd55d1f4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55815cdd0 con 0x7fd55815bd70 2023-11-04T05:28:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.610+0000 7fd55d1f4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580ce8a0 con 0x7fd558152aa0 2023-11-04T05:28:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.610+0000 7fd55659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 0x7fd5580cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55759e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd55815bd70 0x7fd5580ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55759e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd55815bd70 0x7fd5580ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51666/0 (socket says 172.21.15.3:51666) 2023-11-04T05:28:56.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55759e700 1 -- 172.21.15.3:0/3248657243 learned_addr learned my addr 172.21.15.3:0/3248657243 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:56.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 0x7fd5580cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43632/0 (socket says 172.21.15.3:43632) 2023-11-04T05:28:56.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55659c700 1 -- 172.21.15.3:0/3248657243 >> v2:172.21.15.188:3300/0 conn(0x7fd55815bd70 msgr2=0x7fd5580ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:56.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55659c700 1 --2- 172.21.15.3:0/3248657243 >> v2:172.21.15.188:3300/0 conn(0x7fd55815bd70 0x7fd5580ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55659c700 1 -- 172.21.15.3:0/3248657243 >> v1:172.21.15.188:6789/0 conn(0x7fd5581526c0 legacy=0x7fd5580b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55659c700 1 -- 172.21.15.3:0/3248657243 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55815c250 con 0x7fd558152aa0 2023-11-04T05:28:56.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd55659c700 1 --2- 172.21.15.3:0/3248657243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 0x7fd5580cde20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd548009840 tx=0x7fd548009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6697aa8d441f66f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:56.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd555d9b700 1 -- 172.21.15.3:0/3248657243 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011830 con 0x7fd558152aa0 2023-11-04T05:28:56.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd555d9b700 1 -- 172.21.15.3:0/3248657243 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548011990 con 0x7fd558152aa0 2023-11-04T05:28:56.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.611+0000 7fd555d9b700 1 -- 172.21.15.3:0/3248657243 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011d20 con 0x7fd558152aa0 2023-11-04T05:28:56.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3248657243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 msgr2=0x7fd5580cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3248657243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 0x7fd5580cde20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd548009840 tx=0x7fd548009b50 comp rx=0 tx=0).stop 2023-11-04T05:28:56.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3248657243 shutdown_connections 2023-11-04T05:28:56.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3248657243 >> v2:172.21.15.188:3300/0 conn(0x7fd55815bd70 0x7fd5580ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3248657243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd558152aa0 0x7fd5580cde20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3248657243 >> 172.21.15.3:0/3248657243 conn(0x7fd55806ab20 msgr2=0x7fd55806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:56.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3248657243 shutdown_connections 2023-11-04T05:28:56.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.612+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3248657243 wait complete. 2023-11-04T05:28:56.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.613+0000 7fd55d1f4700 1 Processor -- start 2023-11-04T05:28:56.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.613+0000 7fd55d1f4700 1 -- start start 2023-11-04T05:28:56.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd55d1f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 0x7fd5581f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd55d1f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd55815bd70 0x7fd5580c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd55d1f4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd55815a840 0x7fd55815acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd55d1f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c7c90 con 0x7fd5581526c0 2023-11-04T05:28:56.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd55d1f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55815b230 con 0x7fd55815bd70 2023-11-04T05:28:56.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd55d1f4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c7010 con 0x7fd55815a840 2023-11-04T05:28:56.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 0x7fd5581f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 0x7fd5581f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52580/0 (socket says 172.21.15.3:52580) 2023-11-04T05:28:56.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.614+0000 7fd556d9d700 1 -- 172.21.15.3:0/3144267229 learned_addr learned my addr 172.21.15.3:0/3144267229 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:56.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd55659c700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd55815bd70 0x7fd5580c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd55759e700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd55815a840 0x7fd55815acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd556d9d700 1 -- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd55815a840 msgr2=0x7fd55815acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd556d9d700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd55815a840 0x7fd55815acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd556d9d700 1 -- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd55815bd70 msgr2=0x7fd5580c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd556d9d700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd55815bd70 0x7fd5580c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd556d9d700 1 -- 172.21.15.3:0/3144267229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480094a0 con 0x7fd5581526c0 2023-11-04T05:28:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd55659c700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd55815bd70 0x7fd5580c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:56.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd556d9d700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 0x7fd5581f2f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00c4e0 tx=0x7fd54c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0052d0 con 0x7fd5581526c0 2023-11-04T05:28:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c005430 con 0x7fd5581526c0 2023-11-04T05:28:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0038c0 con 0x7fd5581526c0 2023-11-04T05:28:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580c72f0 con 0x7fd5581526c0 2023-11-04T05:28:56.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.615+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5580c78b0 con 0x7fd5581526c0 2023-11-04T05:28:56.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.616+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55815cad0 con 0x7fd5581526c0 2023-11-04T05:28:56.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.617+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd54c003a20 con 0x7fd5581526c0 2023-11-04T05:28:56.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.617+0000 7fd53f7fe700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd54406af00 0x7fd54406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:56.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.617+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd54c088e30 con 0x7fd5581526c0 2023-11-04T05:28:56.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.617+0000 7fd55659c700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd54406af00 0x7fd54406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:56.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.618+0000 7fd55659c700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd54406af00 0x7fd54406d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd548008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:56.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.622+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd55815cad0 con 0x7fd5581526c0 2023-11-04T05:28:56.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.872+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd55815cad0 con 0x7fd5581526c0 2023-11-04T05:28:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.872+0000 7fd53f7fe700 1 -- 172.21.15.3:0/3144267229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7fd55815cad0 con 0x7fd5581526c0 2023-11-04T05:28:56.875 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:28:56.875 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:28:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd54406af00 msgr2=0x7fd54406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd54406af00 0x7fd54406d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd548008030 comp rx=0 tx=0).stop 2023-11-04T05:28:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 msgr2=0x7fd5581f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 0x7fd5581f2f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00c4e0 tx=0x7fd54c00c8a0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 shutdown_connections 2023-11-04T05:28:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd54406af00 0x7fd54406d3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd55815a840 0x7fd55815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5581526c0 0x7fd5581f2f80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 --2- 172.21.15.3:0/3144267229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd55815bd70 0x7fd5580c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.874+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 >> 172.21.15.3:0/3144267229 conn(0x7fd55806ab20 msgr2=0x7fd5580b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.875+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 shutdown_connections 2023-11-04T05:28:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:56.875+0000 7fd55d1f4700 1 -- 172.21.15.3:0/3144267229 wait complete. 2023-11-04T05:28:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:28:57.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-04T05:28:20.853748+0000', 'modified': '2023-11-04T05:28:44.290398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6837/3609354532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6835', 'nonce': 3609354532}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3609354532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-04T05:28:57.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-04T05:28:57.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-04T05:28:57.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.420+0000 7ff725c0b700 1 Processor -- start 2023-11-04T05:28:57.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.420+0000 7ff725c0b700 1 -- start start 2023-11-04T05:28:57.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff725c0b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff72015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff725c0b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff720071820 0x7ff7200cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff725c0b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff720071590 con 0x7ff720160300 2023-11-04T05:28:57.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff725c0b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72015d0a0 con 0x7ff720071820 2023-11-04T05:28:57.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff725c0b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff720070fd0 con 0x7ff72015bb30 2023-11-04T05:28:57.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff720071820 0x7ff7200cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff720071820 0x7ff7200cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52586/0 (socket says 172.21.15.3:52586) 2023-11-04T05:28:57.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 -- 172.21.15.3:0/1880685124 learned_addr learned my addr 172.21.15.3:0/1880685124 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:57.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/1880685124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff72015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 -- 172.21.15.3:0/1880685124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 msgr2=0x7ff72015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 --2- 172.21.15.3:0/1880685124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff72015cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 -- 172.21.15.3:0/1880685124 >> v1:172.21.15.188:6789/0 conn(0x7ff720160300 legacy=0x7ff7200cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.421+0000 7ff71effd700 1 -- 172.21.15.3:0/1880685124 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200712b0 con 0x7ff720071820 2023-11-04T05:28:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff71effd700 1 --2- 172.21.15.3:0/1880685124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff7200cbf00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098f0 tx=0x7ff710009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c5f4005f167e63 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff71e7fc700 1 -- 172.21.15.3:0/1880685124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710011830 con 0x7ff720071820 2023-11-04T05:28:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff71e7fc700 1 -- 172.21.15.3:0/1880685124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff710011990 con 0x7ff720071820 2023-11-04T05:28:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff71e7fc700 1 -- 172.21.15.3:0/1880685124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710011d20 con 0x7ff720071820 2023-11-04T05:28:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 -- 172.21.15.3:0/1880685124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 msgr2=0x7ff7200cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1880685124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff7200cbf00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098f0 tx=0x7ff710009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 -- 172.21.15.3:0/1880685124 shutdown_connections 2023-11-04T05:28:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1880685124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff72015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1880685124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff7200cbf00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 -- 172.21.15.3:0/1880685124 >> 172.21.15.3:0/1880685124 conn(0x7ff72006b4d0 msgr2=0x7ff72006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 -- 172.21.15.3:0/1880685124 shutdown_connections 2023-11-04T05:28:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.422+0000 7ff725c0b700 1 -- 172.21.15.3:0/1880685124 wait complete. 2023-11-04T05:28:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.424+0000 7ff725c0b700 1 Processor -- start 2023-11-04T05:28:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.424+0000 7ff725c0b700 1 -- start start 2023-11-04T05:28:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.424+0000 7ff725c0b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff720156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff7200cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff720160300 0x7ff7200cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200ad690 con 0x7ff720071820 2023-11-04T05:28:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201568a0 con 0x7ff720160300 2023-11-04T05:28:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200aca10 con 0x7ff72015bb30 2023-11-04T05:28:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff720156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff720156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52592/0 (socket says 172.21.15.3:52592) 2023-11-04T05:28:57.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 -- 172.21.15.3:0/1911493315 learned_addr learned my addr 172.21.15.3:0/1911493315 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:57.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 -- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 msgr2=0x7ff7200cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff7200cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71ffff700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff720160300 0x7ff7200cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 -- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff720160300 msgr2=0x7ff7200cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff720160300 0x7ff7200cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 -- 172.21.15.3:0/1911493315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009550 con 0x7ff720071820 2023-11-04T05:28:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff720156360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff70800e9b0 tx=0x7ff70800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff717fff700 1 -- 172.21.15.3:0/1911493315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800c850 con 0x7ff720071820 2023-11-04T05:28:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff717fff700 1 -- 172.21.15.3:0/1911493315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70800c9b0 con 0x7ff720071820 2023-11-04T05:28:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7200accf0 con 0x7ff720071820 2023-11-04T05:28:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff717fff700 1 -- 172.21.15.3:0/1911493315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800cd20 con 0x7ff720071820 2023-11-04T05:28:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.425+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7200ad2b0 con 0x7ff720071820 2023-11-04T05:28:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.427+0000 7ff717fff700 1 -- 172.21.15.3:0/1911493315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff708013470 con 0x7ff720071820 2023-11-04T05:28:57.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.427+0000 7ff717fff700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff70006af00 0x7ff70006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.428+0000 7ff717fff700 1 -- 172.21.15.3:0/1911493315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff708089a10 con 0x7ff720071820 2023-11-04T05:28:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.428+0000 7ff71effd700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff70006af00 0x7ff70006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.428+0000 7ff71effd700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff70006af00 0x7ff70006d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff720071c60 tx=0x7ff710010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.429+0000 7ff731e05b80 1 Processor -- start 2023-11-04T05:28:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.429+0000 7ff731e05b80 1 -- start start 2023-11-04T05:28:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e6194450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6166c30 0x55e5e6194990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e617ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff731e05b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e60d06e0 con 0x55e5e6164ad0 2023-11-04T05:28:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e60c43b0 con 0x55e5e6166c30 2023-11-04T05:28:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e60c1cd0 con 0x55e5e61646f0 2023-11-04T05:28:57.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e617ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e617ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52604/0 (socket says 172.21.15.3:52604) 2023-11-04T05:28:57.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff71ffff700 1 -- 172.21.15.3:0/1215971659 learned_addr learned my addr 172.21.15.3:0/1215971659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:57.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e6194450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.430+0000 7ff71effd700 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6166c30 0x55e5e6194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff71ffff700 1 -- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 msgr2=0x55e5e6194450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff71ffff700 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e6194450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff71ffff700 1 -- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6166c30 msgr2=0x55e5e6194990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff71ffff700 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6166c30 0x55e5e6194990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff71ffff700 1 -- 172.21.15.3:0/1215971659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e5f3a5e0 con 0x55e5e6164ad0 2023-11-04T05:28:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff71ffff700 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e617ef90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff718002030 tx=0x7ff71800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd5f7633174e35a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff7157fa700 1 -- 172.21.15.3:0/1215971659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71800dda0 con 0x55e5e6164ad0 2023-11-04T05:28:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff7157fa700 1 -- 172.21.15.3:0/1215971659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7180123f0 con 0x55e5e6164ad0 2023-11-04T05:28:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.431+0000 7ff7157fa700 1 -- 172.21.15.3:0/1215971659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718012760 con 0x55e5e6164ad0 2023-11-04T05:28:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 -- 172.21.15.3:0/1215971659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 msgr2=0x55e5e617ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e617ef90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff718002030 tx=0x7ff71800d430 comp rx=0 tx=0).stop 2023-11-04T05:28:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 -- 172.21.15.3:0/1215971659 shutdown_connections 2023-11-04T05:28:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e617ef90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e6194450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 --2- 172.21.15.3:0/1215971659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6166c30 0x55e5e6194990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 -- 172.21.15.3:0/1215971659 >> 172.21.15.3:0/1215971659 conn(0x55e5e60df660 msgr2=0x55e5e616a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 -- 172.21.15.3:0/1215971659 shutdown_connections 2023-11-04T05:28:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 -- 172.21.15.3:0/1215971659 wait complete. 2023-11-04T05:28:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 -- start start 2023-11-04T05:28:57.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.432+0000 7ff731e05b80 1 Processor -- start 2023-11-04T05:28:57.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 0x55e5e6191a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6193d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e60bd5c0 con 0x55e5e61646f0 2023-11-04T05:28:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e60590d0 con 0x55e5e6193910 2023-11-04T05:28:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e6058d70 con 0x55e5e6164ad0 2023-11-04T05:28:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff731e05b80 10 client.0 _ll_register_callbacks cb 0x55e5e5eb57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-04T05:28:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 0x55e5e6191a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 0x55e5e6191a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52606/0 (socket says 172.21.15.3:52606) 2023-11-04T05:28:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff71f7fe700 1 -- 172.21.15.3:0/3961343352 learned_addr learned my addr 172.21.15.3:0/3961343352 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff71effd700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.433+0000 7ff71ffff700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6193d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff71f7fe700 1 -- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 msgr2=0x55e5e61a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff71f7fe700 1 -- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 msgr2=0x55e5e6193d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6193d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff71f7fe700 1 -- 172.21.15.3:0/3961343352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff720071590 con 0x55e5e61646f0 2023-11-04T05:28:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 0x55e5e6191a90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff708058410 tx=0x7ff708058500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d924bd9291a43312 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff6fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-11-04T05:28:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/3961343352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708060780 con 0x55e5e61646f0 2023-11-04T05:28:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/3961343352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff70805c1b0 con 0x55e5e61646f0 2023-11-04T05:28:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5e615c880 con 0x55e5e61646f0 2023-11-04T05:28:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/3961343352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70805c370 con 0x55e5e61646f0 2023-11-04T05:28:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 10 client.4860 fetch_fsmap learned FSMap version 9 2023-11-04T05:28:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55e5e5ee6d90 con 0x55e5e61646f0 2023-11-04T05:28:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/3961343352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e5e615c880 con 0x55e5e61646f0 2023-11-04T05:28:57.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/3961343352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff708057970 con 0x55e5e61646f0 2023-11-04T05:28:57.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 10 client.4860 fetch_fsmap finished waiting for FSMap version 9 2023-11-04T05:28:57.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 10 client.4860 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-04T05:28:57.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 20 client.4860 populate_metadata read hostname 'smithi003' 2023-11-04T05:28:57.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.434+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e6195fb0 0x55e5e61bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.435+0000 7ff731e05b80 4 client.4860 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-04T05:28:57.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.435+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 --> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5e6138a30 con 0x55e5e6195fb0 2023-11-04T05:28:57.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.435+0000 7ff71effd700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e6195fb0 0x55e5e61bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.435+0000 7ff6fe7fc700 10 client.4860 ms_handle_connect on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:57.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.435+0000 7ff71effd700 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e6195fb0 0x55e5e61bea70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d1d4fbbaa73a2 server_cookie=5d8cb6c80c36d604 in_seq=0 out_seq=0 2023-11-04T05:28:57.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/3961343352 <== mds.0 v2:172.21.15.188:6835/3609354532 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5e6138a30 con 0x55e5e6195fb0 2023-11-04T05:28:57.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff6fe7fc700 10 client.4860 handle_command_reply: tid=0 2023-11-04T05:28:57.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 client.4860 shutdown 2023-11-04T05:28:57.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 msgr2=0x55e5e6191a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 0x55e5e6191a90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff708058410 tx=0x7ff708058500 comp rx=0 tx=0).stop 2023-11-04T05:28:57.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 shutdown_connections 2023-11-04T05:28:57.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e6195fb0 0x55e5e61bea70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6193d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff6fe7fc700 0 client.4860 ms_handle_reset on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:57.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e61646f0 0x55e5e6191a90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3961343352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 >> 172.21.15.3:0/3961343352 conn(0x55e5e60df660 msgr2=0x55e5e6165020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:57.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 shutdown_connections 2023-11-04T05:28:57.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 1 -- 172.21.15.3:0/3961343352 wait complete. 2023-11-04T05:28:57.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.436+0000 7ff731e05b80 20 client.4860 trim_cache size 0 max 16384 2023-11-04T05:28:57.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 Processor -- start 2023-11-04T05:28:57.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 -- start start 2023-11-04T05:28:57.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e61646f0 0x55e5e61c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 0x55e5e61c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e5f3c8d0 con 0x55e5e6193910 2023-11-04T05:28:57.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e6165b70 con 0x55e5e61646f0 2023-11-04T05:28:57.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.455+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e6166830 con 0x55e5e6164ad0 2023-11-04T05:28:57.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 0x55e5e61c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 0x55e5e61c1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52618/0 (socket says 172.21.15.3:52618) 2023-11-04T05:28:57.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 -- 172.21.15.3:0/3797331654 learned_addr learned my addr 172.21.15.3:0/3797331654 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:57.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71effd700 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e61646f0 0x55e5e61c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 -- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 msgr2=0x55e5e61c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61c0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 -- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e61646f0 msgr2=0x55e5e61c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e61646f0 0x55e5e61c0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 -- 172.21.15.3:0/3797331654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e5ecfc10 con 0x55e5e6193910 2023-11-04T05:28:57.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71effd700 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:57.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff71ffff700 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 0x55e5e61c1500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff718015040 tx=0x7ff718005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8aa74e8fb1bc08d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff715ffb700 1 -- 172.21.15.3:0/3797331654 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718021040 con 0x55e5e6193910 2023-11-04T05:28:57.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff715ffb700 1 -- 172.21.15.3:0/3797331654 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718001670 con 0x55e5e6193910 2023-11-04T05:28:57.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.456+0000 7ff715ffb700 1 -- 172.21.15.3:0/3797331654 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718003a00 con 0x55e5e6193910 2023-11-04T05:28:57.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 -- 172.21.15.3:0/3797331654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 msgr2=0x55e5e61c1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 0x55e5e61c1500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff718015040 tx=0x7ff718005040 comp rx=0 tx=0).stop 2023-11-04T05:28:57.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 -- 172.21.15.3:0/3797331654 shutdown_connections 2023-11-04T05:28:57.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e61646f0 0x55e5e61c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6193910 0x55e5e61c1500 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 --2- 172.21.15.3:0/3797331654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e6164ad0 0x55e5e61c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 -- 172.21.15.3:0/3797331654 >> 172.21.15.3:0/3797331654 conn(0x55e5e60df660 msgr2=0x55e5e61607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:57.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 -- 172.21.15.3:0/3797331654 shutdown_connections 2023-11-04T05:28:57.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 -- 172.21.15.3:0/3797331654 wait complete. 2023-11-04T05:28:57.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.457+0000 7ff731e05b80 1 -- start start 2023-11-04T05:28:57.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.458+0000 7ff731e05b80 1 Processor -- start 2023-11-04T05:28:57.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e616b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e6196390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6163d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e6196e80 con 0x55e5e6164ad0 2023-11-04T05:28:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e61a5200 con 0x55e5e6193910 2023-11-04T05:28:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e61b28d0 con 0x55e5e61646f0 2023-11-04T05:28:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff731e05b80 10 client.0 _ll_register_callbacks cb 0x7ff708057e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-04T05:28:57.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e6196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e6196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52622/0 (socket says 172.21.15.3:52622) 2023-11-04T05:28:57.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 -- 172.21.15.3:0/2938695012 learned_addr learned my addr 172.21.15.3:0/2938695012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:57.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e616b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71ffff700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 -- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 msgr2=0x55e5e616b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e616b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 -- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 msgr2=0x55e5e6163d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:57.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6163d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 -- 172.21.15.3:0/2938695012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e5f3a5e0 con 0x55e5e6164ad0 2023-11-04T05:28:57.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff71effd700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e6196390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff710013000 tx=0x7ff7100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a900f1c658330e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:57.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.459+0000 7ff6fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-11-04T05:28:57.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/2938695012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710003bf0 con 0x55e5e6164ad0 2023-11-04T05:28:57.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/2938695012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff710003d50 con 0x55e5e6164ad0 2023-11-04T05:28:57.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5e6195050 con 0x55e5e6164ad0 2023-11-04T05:28:57.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/2938695012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff710003f10 con 0x55e5e6164ad0 2023-11-04T05:28:57.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 10 client.4866 fetch_fsmap learned FSMap version 9 2023-11-04T05:28:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55e5e5f0d200 con 0x55e5e6164ad0 2023-11-04T05:28:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/2938695012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e5e6195050 con 0x55e5e6164ad0 2023-11-04T05:28:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/2938695012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff710034040 con 0x55e5e6164ad0 2023-11-04T05:28:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 10 client.4866 fetch_fsmap finished waiting for FSMap version 9 2023-11-04T05:28:57.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 10 client.4866 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-04T05:28:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 20 client.4866 populate_metadata read hostname 'smithi003' 2023-11-04T05:28:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e61c0c90 0x55e5e61c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 4 client.4866 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-04T05:28:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 --> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7ff7180123f0 con 0x55e5e61c0c90 2023-11-04T05:28:57.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.460+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e61c0c90 0x55e5e61c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff71f7fe700 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e61c0c90 0x55e5e61c1070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=213fcbb341ae2025 server_cookie=d179b64bd57afd54 in_seq=0 out_seq=0 2023-11-04T05:28:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff6fe7fc700 10 client.4866 ms_handle_connect on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff6fe7fc700 1 -- 172.21.15.3:0/2938695012 <== mds.0 v2:172.21.15.188:6835/3609354532 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7ff7180123f0 con 0x55e5e61c0c90 2023-11-04T05:28:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff6fe7fc700 10 client.4866 handle_command_reply: tid=0 2023-11-04T05:28:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 client.4866 shutdown 2023-11-04T05:28:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 msgr2=0x55e5e6196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e6196390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff710013000 tx=0x7ff7100047c0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 shutdown_connections 2023-11-04T05:28:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:6835/3609354532,v1:172.21.15.188:6837/3609354532] conn(0x55e5e61c0c90 0x55e5e61c1070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5e6193910 0x55e5e6163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5e61646f0 0x55e5e616b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff6fe7fc700 0 client.4866 ms_handle_reset on v2:172.21.15.188:6835/3609354532 2023-11-04T05:28:57.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 --2- 172.21.15.3:0/2938695012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5e6164ad0 0x55e5e6196390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 >> 172.21.15.3:0/2938695012 conn(0x55e5e60df660 msgr2=0x55e5e6169a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:57.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 shutdown_connections 2023-11-04T05:28:57.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 1 -- 172.21.15.3:0/2938695012 wait complete. 2023-11-04T05:28:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.461+0000 7ff731e05b80 20 client.4866 trim_cache size 0 max 16384 2023-11-04T05:28:57.483 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-11-04T05:28:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "7a64e299-05b0-4fd0-819f-2262a06846bb", 2023-11-04T05:28:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-11-04T05:28:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4288, 2023-11-04T05:28:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-11-04T05:28:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-11-04T05:28:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-11-04T05:28:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 36.028169669, 2023-11-04T05:28:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 9, 2023-11-04T05:28:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 26, 2023-11-04T05:28:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 26, 2023-11-04T05:28:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 41.110280721999999 2023-11-04T05:28:57.485 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-11-04T05:28:57.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff70006af00 msgr2=0x7ff70006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff70006af00 0x7ff70006d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff720071c60 tx=0x7ff710010000 comp rx=0 tx=0).stop 2023-11-04T05:28:57.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 msgr2=0x7ff720156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:57.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff720156360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff70800e9b0 tx=0x7ff70800ed70 comp rx=0 tx=0).stop 2023-11-04T05:28:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 shutdown_connections 2023-11-04T05:28:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff70006af00 0x7ff70006d3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff720160300 0x7ff7200cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff720071820 0x7ff720156360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 --2- 172.21.15.3:0/1911493315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72015bb30 0x7ff7200cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 >> 172.21.15.3:0/1911493315 conn(0x7ff72006b4d0 msgr2=0x7ff7200c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 shutdown_connections 2023-11-04T05:28:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:57.462+0000 7ff725c0b700 1 -- 172.21.15.3:0/1911493315 wait complete. 2023-11-04T05:28:57.489 INFO:teuthology.run_tasks:Running task kclient... 2023-11-04T05:28:57.503 INFO:tasks.kclient:Mounting kernel clients... 2023-11-04T05:28:57.503 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-04T05:28:57.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.504 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-11-04T05:28:57.522 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-04T05:28:57.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-04T05:28:57.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-11-04T05:28:57.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.725 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-11-04T05:28:57.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-11-04T05:28:57.812 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2023-11-04T05:28:57.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:28:57.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.814 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-04T05:28:57.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.836 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-04T05:28:57.908 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2023-11-04T05:28:57.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:28:57.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.911 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-04T05:28:57.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:57.969 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-04T05:28:58.039 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2023-11-04T05:28:58.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T05:28:58.040 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-04T05:28:58.041 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-04T05:28:58.041 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-04T05:28:58.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-04T05:28:58.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.167+0000 7f8d234b8700 1 Processor -- start 2023-11-04T05:28:58.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.167+0000 7f8d234b8700 1 -- start start 2023-11-04T05:28:58.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d234b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:58.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d234b8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8d1c0cd6d0 0x7f8d1c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:58.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d234b8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c15df70 con 0x7f8d1c0cf610 2023-11-04T05:28:58.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d234b8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0d1690 con 0x7f8d1c0cd6d0 2023-11-04T05:28:58.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d234b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c06d800 con 0x7f8d1c0cf230 2023-11-04T05:28:58.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8d1c0cd6d0 0x7f8d1c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:58.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d21254700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:58.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8d1c0cd6d0 0x7f8d1c06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:51716/0 (socket says 172.21.15.3:51716) 2023-11-04T05:28:58.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d21a55700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8d1c0cf610 0x7f8d1c161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:36520/0 (socket says 172.21.15.3:36520) 2023-11-04T05:28:58.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 -- 172.21.15.3:0/153883858 learned_addr learned my addr 172.21.15.3:0/153883858 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:58.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 -- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 msgr2=0x7f8d1c0d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:28:58.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 --2- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 -- 172.21.15.3:0/153883858 >> v1:172.21.15.188:6789/0 conn(0x7f8d1c0cf610 legacy=0x7f8d1c161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:58.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d20a53700 1 -- 172.21.15.3:0/153883858 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0d1930 con 0x7f8d1c0cd6d0 2023-11-04T05:28:58.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d13fff700 1 -- 172.21.15.3:0/153883858 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082620001 0 0) 0x7f8d1c15df70 con 0x7f8d1c0cf610 2023-11-04T05:28:58.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.168+0000 7f8d21254700 1 --2- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:28:58.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d20a53700 1 --2- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c06caf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098f0 tx=0x7f8d0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c1c9bbf241380b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:58.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d13fff700 1 -- 172.21.15.3:0/153883858 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c011830 con 0x7f8d1c0cd6d0 2023-11-04T05:28:58.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d13fff700 1 -- 172.21.15.3:0/153883858 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c011990 con 0x7f8d1c0cd6d0 2023-11-04T05:28:58.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d13fff700 1 -- 172.21.15.3:0/153883858 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c011d20 con 0x7f8d1c0cd6d0 2023-11-04T05:28:58.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 -- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 msgr2=0x7f8d1c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:58.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 --2- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c06caf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098f0 tx=0x7f8d0c009c00 comp rx=0 tx=0).stop 2023-11-04T05:28:58.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 -- 172.21.15.3:0/153883858 shutdown_connections 2023-11-04T05:28:58.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 --2- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 --2- 172.21.15.3:0/153883858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c06caf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 -- 172.21.15.3:0/153883858 >> 172.21.15.3:0/153883858 conn(0x7f8d1c06ab20 msgr2=0x7f8d1c06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:58.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 -- 172.21.15.3:0/153883858 shutdown_connections 2023-11-04T05:28:58.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.169+0000 7f8d234b8700 1 -- 172.21.15.3:0/153883858 wait complete. 2023-11-04T05:28:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.171+0000 7f8d234b8700 1 Processor -- start 2023-11-04T05:28:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.171+0000 7f8d234b8700 1 -- start start 2023-11-04T05:28:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.171+0000 7f8d234b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.171+0000 7f8d234b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.171+0000 7f8d234b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 0x7f8d1c0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d234b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0c9a60 con 0x7f8d1c0cf610 2023-11-04T05:28:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d234b8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c159e00 con 0x7f8d1c0cd6d0 2023-11-04T05:28:58.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d234b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b6830 con 0x7f8d1c0cf230 2023-11-04T05:28:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 0x7f8d1c0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 0x7f8d1c0c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52638/0 (socket says 172.21.15.3:52638) 2023-11-04T05:28:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 -- 172.21.15.3:0/2752720904 learned_addr learned my addr 172.21.15.3:0/2752720904 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:28:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d20a53700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21254700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 -- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 msgr2=0x7f8d1c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 -- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 msgr2=0x7f8d1c1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c1598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 -- 172.21.15.3:0/2752720904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180094a0 con 0x7f8d1c0cf610 2023-11-04T05:28:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21254700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:28:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d21a55700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 0x7f8d1c0c5a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8d14002230 tx=0x7f8d1400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400ab70 con 0x7f8d1c0cf610 2023-11-04T05:28:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400acf0 con 0x7f8d1c0cf610 2023-11-04T05:28:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c009550 con 0x7f8d1c0cf610 2023-11-04T05:28:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.172+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14007310 con 0x7f8d1c0cf610 2023-11-04T05:28:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.173+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c0c9cb0 con 0x7f8d1c0cf610 2023-11-04T05:28:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.174+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c0b3380 con 0x7f8d1c0cf610 2023-11-04T05:28:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.174+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d14007470 con 0x7f8d1c0cf610 2023-11-04T05:28:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.174+0000 7f8d11ffb700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8d0006af00 0x7f8d0006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:28:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.174+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8d14089690 con 0x7f8d1c0cf610 2023-11-04T05:28:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.175+0000 7f8d21254700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8d0006af00 0x7f8d0006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:28:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.175+0000 7f8d21254700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8d0006af00 0x7f8d0006d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0b70d0 tx=0x7f8d18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:28:58.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.179+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8d140544e0 con 0x7f8d1c0cf610 2023-11-04T05:28:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.425+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8d1c074df0 con 0x7f8d1c0cf610 2023-11-04T05:28:58.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.425+0000 7f8d11ffb700 1 -- 172.21.15.3:0/2752720904 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8d14018070 con 0x7f8d1c0cf610 2023-11-04T05:28:58.428 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-04T05:28:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.427+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8d0006af00 msgr2=0x7f8d0006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:58.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.427+0000 7f8d234b8700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8d0006af00 0x7f8d0006d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0b70d0 tx=0x7f8d18008040 comp rx=0 tx=0).stop 2023-11-04T05:28:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.427+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 msgr2=0x7f8d1c0c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:28:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.427+0000 7f8d234b8700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 0x7f8d1c0c5a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8d14002230 tx=0x7f8d1400cf50 comp rx=0 tx=0).stop 2023-11-04T05:28:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 shutdown_connections 2023-11-04T05:28:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8d0006af00 0x7f8d0006d3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0cd6d0 0x7f8d1c1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d1c0cf610 0x7f8d1c0c5a00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 --2- 172.21.15.3:0/2752720904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0cf230 0x7f8d1c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:28:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 >> 172.21.15.3:0/2752720904 conn(0x7f8d1c06ab20 msgr2=0x7f8d1c0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:28:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 shutdown_connections 2023-11-04T05:28:58.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:28:58.428+0000 7f8d234b8700 1 -- 172.21.15.3:0/2752720904 wait complete. 2023-11-04T05:28:58.442 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-04T05:28:58.442 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-04T05:28:58.443 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-11-04T05:28:58.443 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-04T05:28:58.443 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-04T05:28:58.443 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-04T05:28:58.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-04T05:28:58.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-04T05:28:58.444 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-04T05:28:58.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.445 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-11-04T05:28:58.462 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:28:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:28:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:28:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:28:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-11-04T05:28:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:28:58.464 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:28:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:58.465 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:28:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:58.465 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-04T05:28:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-04T05:28:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-11-04T05:28:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-11-04T05:28:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:28:58.467 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:28:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-11-04T05:28:58.467 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-04T05:28:58.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.468 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:58.468 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:58.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-11-04T05:28:58.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-11-04T05:28:58.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-11-04T05:28:58.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-04T05:28:58.470 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:58.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.583 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-04T05:28:58.610 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-11-04T05:28:58.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.612 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-11-04T05:28:58.671 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-11-04T05:28:58.671 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-04T05:28:58.672 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-04T05:28:58.672 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-04T05:28:58.672 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-04T05:28:58.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.673 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:58.673 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:58.673 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-04T05:28:58.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-04T05:28:58.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-04T05:28:58.674 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:58.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.780 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-11-04T05:28:58.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.798 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-11-04T05:28:58.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.855 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:58.856 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:58.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-04T05:28:58.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-04T05:28:58.856 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:58.950 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-04T05:28:58.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:58.951 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:58.952 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:58.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-04T05:28:58.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-04T05:28:58.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-04T05:28:58.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-04T05:28:58.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-04T05:28:58.953 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:59.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.168 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:28:59.168 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:28:59.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-11-04T05:28:59.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-11-04T05:28:59.169 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:28:59.296 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:28:59.297 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-04T05:28:59.315 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-04T05:28:59.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:28:59.316 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-04T05:28:59.371 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-04T05:28:59.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:28:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:28:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-04T05:28:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:28:59.586 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:28:59.586 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-04T05:28:59.586 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:28:59.586 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:28:59.587 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:28:59.619 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:28:59.619 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-04T05:28:59.619 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:28:59.619 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:28:59.620 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:28:59.620 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790" 2023-11-04T05:28:59.620 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2023-11-04T05:28:59.620 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb" will pass to kernel 2023-11-04T05:28:59.626 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-04T05:28:59.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-04T05:28:59.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.664 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:28:59.737 INFO:teuthology.orchestra.run.smithi003.stdout:client4872 2023-11-04T05:28:59.738 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:28:59.738 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:28:59.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.796 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:28:59.872 INFO:teuthology.orchestra.run.smithi003.stdout:client4872 2023-11-04T05:28:59.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.873 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:28:59.944 INFO:teuthology.orchestra.run.smithi003.stdout:client4872 2023-11-04T05:28:59.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:28:59.945 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7a64e299-05b0-4fd0-819f-2262a06846bb.client4872/status') 2023-11-04T05:29:00.026 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4872 (3)192.168.0.1:0/90771037 2023-11-04T05:29:00.027 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2023-11-04T05:29:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2023-11-04T05:29:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2023-11-04T05:29:00.028 INFO:teuthology.orchestra.run.smithi003.stderr:67 bytes copied, 2.6768e-05 s, 2.5 MB/s 2023-11-04T05:29:00.029 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-04T05:29:00.029 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-04T05:29:00.029 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-04T05:29:00.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-04T05:29:00.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.191+0000 7f4ece25b700 1 Processor -- start 2023-11-04T05:29:00.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.191+0000 7f4ece25b700 1 -- start start 2023-11-04T05:29:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ece25b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ec815da80 0x7f4ec80d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ece25b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ec80cf500 0x7f4ec80d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ece25b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec80d14e0 con 0x7f4ec80d2370 2023-11-04T05:29:00.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ece25b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec806bce0 con 0x7f4ec80cf500 2023-11-04T05:29:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ece25b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec80d27e0 con 0x7f4ec815da80 2023-11-04T05:29:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ec6ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ec80cf500 0x7f4ec80d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ec7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4ec80d2370 0x7f4ec80ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57998/0 (socket says 172.21.15.3:57998) 2023-11-04T05:29:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ec7fff700 1 -- 172.21.15.3:0/887335297 learned_addr learned my addr 172.21.15.3:0/887335297 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.192+0000 7f4ec77fe700 1 --2- 172.21.15.3:0/887335297 >> v2:172.21.15.188:3300/0 conn(0x7f4ec815da80 0x7f4ec80d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:00.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec6ffd700 1 -- 172.21.15.3:0/887335297 >> v2:172.21.15.188:3300/0 conn(0x7f4ec815da80 msgr2=0x7f4ec80d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:00.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec6ffd700 1 --2- 172.21.15.3:0/887335297 >> v2:172.21.15.188:3300/0 conn(0x7f4ec815da80 0x7f4ec80d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec6ffd700 1 -- 172.21.15.3:0/887335297 >> v1:172.21.15.3:6789/0 conn(0x7f4ec80d2370 legacy=0x7f4ec80ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:00.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec6ffd700 1 -- 172.21.15.3:0/887335297 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80d1780 con 0x7f4ec80cf500 2023-11-04T05:29:00.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec67fc700 1 -- 172.21.15.3:0/887335297 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384322272 0 0) 0x7f4ec80d14e0 con 0x7f4ec80d2370 2023-11-04T05:29:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec6ffd700 1 --2- 172.21.15.3:0/887335297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec80d1e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80d4110 tx=0x7f4ebc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cb037b9779fff7 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec67fc700 1 -- 172.21.15.3:0/887335297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011830 con 0x7f4ec80cf500 2023-11-04T05:29:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec67fc700 1 -- 172.21.15.3:0/887335297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc011990 con 0x7f4ec80cf500 2023-11-04T05:29:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.193+0000 7f4ec67fc700 1 -- 172.21.15.3:0/887335297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011d20 con 0x7f4ec80cf500 2023-11-04T05:29:00.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 -- 172.21.15.3:0/887335297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 msgr2=0x7f4ec80d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 --2- 172.21.15.3:0/887335297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec80d1e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80d4110 tx=0x7f4ebc009b20 comp rx=0 tx=0).stop 2023-11-04T05:29:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 -- 172.21.15.3:0/887335297 shutdown_connections 2023-11-04T05:29:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 --2- 172.21.15.3:0/887335297 >> v2:172.21.15.188:3300/0 conn(0x7f4ec815da80 0x7f4ec80d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 --2- 172.21.15.3:0/887335297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec80d1e30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 -- 172.21.15.3:0/887335297 >> 172.21.15.3:0/887335297 conn(0x7f4ec808aff0 msgr2=0x7f4ec80d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 -- 172.21.15.3:0/887335297 shutdown_connections 2023-11-04T05:29:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.194+0000 7f4ece25b700 1 -- 172.21.15.3:0/887335297 wait complete. 2023-11-04T05:29:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.195+0000 7f4ece25b700 1 Processor -- start 2023-11-04T05:29:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.195+0000 7f4ece25b700 1 -- start start 2023-11-04T05:29:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ece25b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec809ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ece25b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d2370 0x7f4ec80a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ece25b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec815da80 0x7f4ec81e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ece25b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80a1630 con 0x7f4ec80cf500 2023-11-04T05:29:00.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ece25b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec809f4b0 con 0x7f4ec815da80 2023-11-04T05:29:00.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ece25b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80a1350 con 0x7f4ec80d2370 2023-11-04T05:29:00.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec809ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:00.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec809ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52652/0 (socket says 172.21.15.3:52652) 2023-11-04T05:29:00.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 -- 172.21.15.3:0/820887146 learned_addr learned my addr 172.21.15.3:0/820887146 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:00.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec6ffd700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d2370 0x7f4ec80a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:00.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec7fff700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec815da80 0x7f4ec81e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:00.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 -- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d2370 msgr2=0x7f4ec80a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:00.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d2370 0x7f4ec80a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 -- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec815da80 msgr2=0x7f4ec81e73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec815da80 0x7f4ec81e73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 -- 172.21.15.3:0/820887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0094a0 con 0x7f4ec80cf500 2023-11-04T05:29:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec7fff700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec815da80 0x7f4ec81e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.196+0000 7f4ec77fe700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec809ef70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e730 tx=0x7f4eb800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.197+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8013610 con 0x7f4ec80cf500 2023-11-04T05:29:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.197+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8013770 con 0x7f4ec80cf500 2023-11-04T05:29:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.197+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec81f36d0 con 0x7f4ec80cf500 2023-11-04T05:29:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.197+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8013b20 con 0x7f4ec80cf500 2023-11-04T05:29:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.197+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec81f3c90 con 0x7f4ec80cf500 2023-11-04T05:29:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.198+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec81f3e50 con 0x7f4ec80cf500 2023-11-04T05:29:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.198+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4eb8013c80 con 0x7f4ec80cf500 2023-11-04T05:29:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.198+0000 7f4eaffff700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4eb406af00 0x7f4eb406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.198+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4eb80891e0 con 0x7f4ec80cf500 2023-11-04T05:29:00.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.199+0000 7f4ec6ffd700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4eb406af00 0x7f4eb406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.199+0000 7f4ec6ffd700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4eb406af00 0x7f4eb406d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80cf940 tx=0x7f4ebc01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.203+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4eb8054030 con 0x7f4ec80cf500 2023-11-04T05:29:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.451+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ec8074df0 con 0x7f4ec80cf500 2023-11-04T05:29:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.452+0000 7f4eaffff700 1 -- 172.21.15.3:0/820887146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4eb8057650 con 0x7f4ec80cf500 2023-11-04T05:29:00.454 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-04T05:29:00.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4eb406af00 msgr2=0x7f4eb406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:00.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4eb406af00 0x7f4eb406d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80cf940 tx=0x7f4ebc01c040 comp rx=0 tx=0).stop 2023-11-04T05:29:00.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 msgr2=0x7f4ec809ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:00.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec809ef70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e730 tx=0x7f4eb800eaf0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 shutdown_connections 2023-11-04T05:29:00.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4eb406af00 0x7f4eb406d3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec815da80 0x7f4ec81e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ec80cf500 0x7f4ec809ef70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 --2- 172.21.15.3:0/820887146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d2370 0x7f4ec80a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:00.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 >> 172.21.15.3:0/820887146 conn(0x7f4ec808aff0 msgr2=0x7f4ec815ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:00.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 shutdown_connections 2023-11-04T05:29:00.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:00.454+0000 7f4ece25b700 1 -- 172.21.15.3:0/820887146 wait complete. 2023-11-04T05:29:00.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-04T05:29:00.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-04T05:29:00.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-11-04T05:29:00.469 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-04T05:29:00.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-04T05:29:00.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-04T05:29:00.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-04T05:29:00.471 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-04T05:29:00.471 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-04T05:29:00.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.472 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-11-04T05:29:00.489 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:00.490 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:00.490 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:00.490 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.490 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:00.491 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.491 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:00.491 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:00.491 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:00.492 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:00.492 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-04T05:29:00.492 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:00.492 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-04T05:29:00.493 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42365sec preferred_lft 42365sec 2023-11-04T05:29:00.493 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-11-04T05:29:00.493 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.493 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:00.493 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:00.494 INFO:teuthology.orchestra.run.smithi003.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:00.494 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether da:2d:22:80:c4:4a brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:00.494 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-04T05:29:00.494 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.495 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::8f4:7fff:fea9:c2dc/64 scope link 2023-11-04T05:29:00.495 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.495 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-04T05:29:00.495 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether da:2d:22:80:c4:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-04T05:29:00.496 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::d82d:22ff:fe80:c44a/64 scope link tentative 2023-11-04T05:29:00.496 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.497 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-04T05:29:00.556 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-11-04T05:29:00.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.558 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-11-04T05:29:00.617 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-11-04T05:29:00.617 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-04T05:29:00.617 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-04T05:29:00.618 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-04T05:29:00.618 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-04T05:29:00.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.619 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:29:00.619 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:29:00.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-04T05:29:00.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-04T05:29:00.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-04T05:29:00.620 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:29:00.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.725 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-11-04T05:29:00.742 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-04T05:29:00.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.743 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-11-04T05:29:00.798 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-04T05:29:00.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.799 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:29:00.800 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:29:00.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-04T05:29:00.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-04T05:29:00.800 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:29:00.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-04T05:29:00.933 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:00.933 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:00.933 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:00.933 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.934 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:00.934 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.934 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:00.934 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 96:34:f0:fa:ce:de brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-04T05:29:00.934 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-04T05:29:00.935 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.935 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::9434:f0ff:fefa:cede/64 scope link tentative 2023-11-04T05:29:00.935 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:00.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:00.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-04T05:29:01.033 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:01.033 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:01.034 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:01.034 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:01.034 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:01.034 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:01.034 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:01.035 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 96:34:f0:fa:ce:de brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-04T05:29:01.035 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-04T05:29:01.035 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:01.035 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::9434:f0ff:fefa:cede/64 scope link tentative 2023-11-04T05:29:01.035 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:01.042 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-04T05:29:01.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.043 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:29:01.043 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:29:01.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-04T05:29:01.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-04T05:29:01.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-04T05:29:01.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-04T05:29:01.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-04T05:29:01.045 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:29:01.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.296 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:29:01.296 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:29:01.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2023-11-04T05:29:01.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2023-11-04T05:29:01.297 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:29:01.429 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:01.429 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-04T05:29:01.455 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-04T05:29:01.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:01.456 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-04T05:29:01.512 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-04T05:29:01.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-04T05:29:01.599 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-04T05:29:01.600 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-04T05:29:01.600 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:29:01.608 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-04T05:29:01.608 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-04T05:29:01.609 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:29:01.609 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:01.609 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:01.636 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-04T05:29:01.637 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-04T05:29:01.637 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:29:01.637 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:01.637 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:01.638 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790" 2023-11-04T05:29:01.638 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2023-11-04T05:29:01.638 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb" will pass to kernel 2023-11-04T05:29:01.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-04T05:29:01.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-04T05:29:01.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.685 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:01.758 INFO:teuthology.orchestra.run.smithi003.stdout:client4884 2023-11-04T05:29:01.759 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:01.759 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:01.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.817 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:01.889 INFO:teuthology.orchestra.run.smithi003.stdout:client4884 2023-11-04T05:29:01.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.890 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:01.963 INFO:teuthology.orchestra.run.smithi003.stdout:client4884 2023-11-04T05:29:01.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:01.964 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7a64e299-05b0-4fd0-819f-2262a06846bb.client4884/status') 2023-11-04T05:29:02.045 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4884 (3)192.168.0.2:0/735183266 2023-11-04T05:29:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2023-11-04T05:29:02.047 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2023-11-04T05:29:02.047 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2023-11-04T05:29:02.047 INFO:teuthology.orchestra.run.smithi003.stderr:68 bytes copied, 2.392e-05 s, 2.8 MB/s 2023-11-04T05:29:02.048 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-04T05:29:02.048 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-04T05:29:02.048 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-04T05:29:02.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-04T05:29:02.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.210+0000 7f5d82acb700 1 Processor -- start 2023-11-04T05:29:02.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.210+0000 7f5d82acb700 1 -- start start 2023-11-04T05:29:02.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d82acb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d7c154550 0x7f5d7c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:02.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d82acb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c06d790 0x7f5d7c06dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:02.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d82acb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c154930 con 0x7f5d7c15dc40 2023-11-04T05:29:02.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d82acb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c06e150 con 0x7f5d7c154550 2023-11-04T05:29:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d82acb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c15e9b0 con 0x7f5d7c06d790 2023-11-04T05:29:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d7c154550 0x7f5d7c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d7c154550 0x7f5d7c0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52664/0 (socket says 172.21.15.3:52664) 2023-11-04T05:29:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 -- 172.21.15.3:0/566922689 learned_addr learned my addr 172.21.15.3:0/566922689 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d7bfff700 1 --2- 172.21.15.3:0/566922689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c06d790 0x7f5d7c06dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 -- 172.21.15.3:0/566922689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c06d790 msgr2=0x7f5d7c06dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:02.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 --2- 172.21.15.3:0/566922689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c06d790 0x7f5d7c06dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 -- 172.21.15.3:0/566922689 >> v1:172.21.15.188:6789/0 conn(0x7f5d7c15dc40 legacy=0x7f5d7c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:02.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 -- 172.21.15.3:0/566922689 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0cddf0 con 0x7f5d7c154550 2023-11-04T05:29:02.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d80867700 1 --2- 172.21.15.3:0/566922689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c154550 0x7f5d7c0cebb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d700098f0 tx=0x7f5d70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce15d3693d813b3b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:02.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.211+0000 7f5d7b7fe700 1 -- 172.21.15.3:0/566922689 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70010800 con 0x7f5d7c154550 2023-11-04T05:29:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d7b7fe700 1 -- 172.21.15.3:0/566922689 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70010960 con 0x7f5d7c154550 2023-11-04T05:29:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d7b7fe700 1 -- 172.21.15.3:0/566922689 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70010cf0 con 0x7f5d7c154550 2023-11-04T05:29:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 -- 172.21.15.3:0/566922689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c154550 msgr2=0x7f5d7c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 --2- 172.21.15.3:0/566922689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c154550 0x7f5d7c0cebb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d700098f0 tx=0x7f5d70009c00 comp rx=0 tx=0).stop 2023-11-04T05:29:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 -- 172.21.15.3:0/566922689 shutdown_connections 2023-11-04T05:29:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 --2- 172.21.15.3:0/566922689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c154550 0x7f5d7c0cebb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 --2- 172.21.15.3:0/566922689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c06d790 0x7f5d7c06dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 -- 172.21.15.3:0/566922689 >> 172.21.15.3:0/566922689 conn(0x7f5d7c0cfd50 msgr2=0x7f5d7c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 -- 172.21.15.3:0/566922689 shutdown_connections 2023-11-04T05:29:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.212+0000 7f5d82acb700 1 -- 172.21.15.3:0/566922689 wait complete. 2023-11-04T05:29:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.214+0000 7f5d82acb700 1 Processor -- start 2023-11-04T05:29:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.214+0000 7f5d82acb700 1 -- start start 2023-11-04T05:29:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d82acb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 0x7f5d7c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d82acb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c154550 0x7f5d7c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d82acb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15dc40 0x7f5d7c0c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d82acb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c154c60 con 0x7f5d7c06d790 2023-11-04T05:29:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d82acb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c154f40 con 0x7f5d7c15dc40 2023-11-04T05:29:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d82acb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c155220 con 0x7f5d7c154550 2023-11-04T05:29:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 0x7f5d7c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 0x7f5d7c155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52670/0 (socket says 172.21.15.3:52670) 2023-11-04T05:29:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 -- 172.21.15.3:0/4120665266 learned_addr learned my addr 172.21.15.3:0/4120665266 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d81068700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15dc40 0x7f5d7c0c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 -- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c154550 msgr2=0x7f5d7c0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c154550 0x7f5d7c0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 -- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15dc40 msgr2=0x7f5d7c0c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15dc40 0x7f5d7c0c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 -- 172.21.15.3:0/4120665266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009550 con 0x7f5d7c06d790 2023-11-04T05:29:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d81068700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15dc40 0x7f5d7c0c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.215+0000 7f5d80867700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 0x7f5d7c155d00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009c40 tx=0x7f5d70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.216+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70003ab0 con 0x7f5d7c06d790 2023-11-04T05:29:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.216+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70003c30 con 0x7f5d7c06d790 2023-11-04T05:29:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.216+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70018400 con 0x7f5d7c06d790 2023-11-04T05:29:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.216+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c155500 con 0x7f5d7c06d790 2023-11-04T05:29:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.216+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c0c6480 con 0x7f5d7c06d790 2023-11-04T05:29:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.217+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c155a20 con 0x7f5d7c06d790 2023-11-04T05:29:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.217+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d70018560 con 0x7f5d7c06d790 2023-11-04T05:29:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.218+0000 7f5d797fa700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5d6006af00 0x7f5d6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.218+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5d7008ac90 con 0x7f5d7c06d790 2023-11-04T05:29:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.219+0000 7f5d7bfff700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5d6006af00 0x7f5d6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.222+0000 7f5d7bfff700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5d6006af00 0x7f5d6006d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006ac0 tx=0x7f5d68006a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.222+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5d7c155a20 con 0x7f5d7c06d790 2023-11-04T05:29:02.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.469+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d7c155a20 con 0x7f5d7c06d790 2023-11-04T05:29:02.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.470+0000 7f5d797fa700 1 -- 172.21.15.3:0/4120665266 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5d7c155a20 con 0x7f5d7c06d790 2023-11-04T05:29:02.472 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-04T05:29:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5d6006af00 msgr2=0x7f5d6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5d6006af00 0x7f5d6006d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006ac0 tx=0x7f5d68006a50 comp rx=0 tx=0).stop 2023-11-04T05:29:02.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 msgr2=0x7f5d7c155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 0x7f5d7c155d00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009c40 tx=0x7f5d70004750 comp rx=0 tx=0).stop 2023-11-04T05:29:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 shutdown_connections 2023-11-04T05:29:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5d6006af00 0x7f5d6006d3c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15dc40 0x7f5d7c0c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d7c06d790 0x7f5d7c155d00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 --2- 172.21.15.3:0/4120665266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c154550 0x7f5d7c0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 >> 172.21.15.3:0/4120665266 conn(0x7f5d7c0cfd50 msgr2=0x7f5d7c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 shutdown_connections 2023-11-04T05:29:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:02.472+0000 7f5d82acb700 1 -- 172.21.15.3:0/4120665266 wait complete. 2023-11-04T05:29:02.486 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-04T05:29:02.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-04T05:29:02.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-04T05:29:02.487 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-04T05:29:02.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-04T05:29:02.488 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-04T05:29:02.488 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-04T05:29:02.488 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-04T05:29:02.489 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-04T05:29:02.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:02.489 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-04T05:29:02.511 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:02.511 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:02.511 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:02.511 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:02.512 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:02.512 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:02.512 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:02.512 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:02.512 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:02.513 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:02.513 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:02.513 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:02.513 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-04T05:29:02.513 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:02.514 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-04T05:29:02.514 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42365sec preferred_lft 42365sec 2023-11-04T05:29:02.514 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::6c9a:ab52:aed8:81ea/64 scope link noprefixroute 2023-11-04T05:29:02.514 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:02.515 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-04T05:29:02.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:02.515 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:02.516 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:02.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2023-11-04T05:29:02.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2023-11-04T05:29:02.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2023-11-04T05:29:02.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-04T05:29:02.517 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:02.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:02.721 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-04T05:29:02.749 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-04T05:29:02.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:02.751 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-04T05:29:02.819 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-04T05:29:02.819 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-04T05:29:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-04T05:29:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-04T05:29:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-04T05:29:02.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:02.821 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:02.821 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:02.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-04T05:29:02.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-04T05:29:02.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-04T05:29:02.822 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:03.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.066 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-04T05:29:03.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.084 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2023-11-04T05:29:03.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.141 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:03.141 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:03.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-04T05:29:03.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-04T05:29:03.142 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:03.235 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-04T05:29:03.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.235 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:03.236 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:03.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-04T05:29:03.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-04T05:29:03.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-04T05:29:03.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-04T05:29:03.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-04T05:29:03.238 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:03.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.439 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:03.439 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:03.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-04T05:29:03.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2023-11-04T05:29:03.440 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:03.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:03.545 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-04T05:29:03.563 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-04T05:29:03.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:03.564 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-04T05:29:03.620 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-04T05:29:03.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-04T05:29:03.715 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-04T05:29:03.715 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-04T05:29:03.715 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2023-11-04T05:29:03.812 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-04T05:29:03.813 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-04T05:29:03.813 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2023-11-04T05:29:03.813 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:03.813 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:03.845 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-04T05:29:03.845 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-04T05:29:03.845 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2023-11-04T05:29:03.845 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:03.846 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:03.846 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790" 2023-11-04T05:29:03.846 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2023-11-04T05:29:03.846 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb" will pass to kernel 2023-11-04T05:29:03.862 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-04T05:29:03.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.863 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-04T05:29:03.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:03.892 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:03.963 INFO:teuthology.orchestra.run.smithi188.stdout:client4580 2023-11-04T05:29:03.964 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:03.964 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:04.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.021 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:04.094 INFO:teuthology.orchestra.run.smithi188.stdout:client4580 2023-11-04T05:29:04.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.095 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:client4580 2023-11-04T05:29:04.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.167 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7a64e299-05b0-4fd0-819f-2262a06846bb.client4580/status') 2023-11-04T05:29:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4580 (3)192.168.0.1:0/1970593551 2023-11-04T05:29:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2023-11-04T05:29:04.248 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2023-11-04T05:29:04.248 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2023-11-04T05:29:04.249 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 2.1266e-05 s, 3.2 MB/s 2023-11-04T05:29:04.249 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-04T05:29:04.249 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-04T05:29:04.250 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-04T05:29:04.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-04T05:29:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.387+0000 7f463a6fc700 1 Processor -- start 2023-11-04T05:29:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 -- start start 2023-11-04T05:29:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46340cf230 0x7f46340d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46340cd6d0 0x7f463406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463415df70 con 0x7f46340cf230 2023-11-04T05:29:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340d1690 con 0x7f46340cd6d0 2023-11-04T05:29:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f463a6fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463406d800 con 0x7f46340cf610 2023-11-04T05:29:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.388+0000 7f4633fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46340cf230 0x7f46340d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46340cf230 0x7f46340d2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52678/0 (socket says 172.21.15.3:52678) 2023-11-04T05:29:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 -- 172.21.15.3:0/3684095336 learned_addr learned my addr 172.21.15.3:0/3684095336 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f46337fe700 1 --2- 172.21.15.3:0/3684095336 >> v2:172.21.15.188:3300/0 conn(0x7f46340cd6d0 0x7f463406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4638c99700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 -- 172.21.15.3:0/3684095336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 msgr2=0x7f4634161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 -- 172.21.15.3:0/3684095336 >> v2:172.21.15.188:3300/0 conn(0x7f46340cd6d0 msgr2=0x7f463406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 --2- 172.21.15.3:0/3684095336 >> v2:172.21.15.188:3300/0 conn(0x7f46340cd6d0 0x7f463406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 -- 172.21.15.3:0/3684095336 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340d1930 con 0x7f46340cf230 2023-11-04T05:29:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4638c99700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4633fff700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cf230 0x7f46340d2130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f46340cee60 tx=0x7f4624009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49836ef801c84db1 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4632ffd700 1 -- 172.21.15.3:0/3684095336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400b870 con 0x7f46340cf230 2023-11-04T05:29:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4632ffd700 1 -- 172.21.15.3:0/3684095336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400b9d0 con 0x7f46340cf230 2023-11-04T05:29:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.389+0000 7f4632ffd700 1 -- 172.21.15.3:0/3684095336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400bd60 con 0x7f46340cf230 2023-11-04T05:29:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 -- 172.21.15.3:0/3684095336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cf230 msgr2=0x7f46340d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cf230 0x7f46340d2130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f46340cee60 tx=0x7f4624009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 -- 172.21.15.3:0/3684095336 shutdown_connections 2023-11-04T05:29:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 --2- 172.21.15.3:0/3684095336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cf230 0x7f46340d2130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 --2- 172.21.15.3:0/3684095336 >> v2:172.21.15.188:3300/0 conn(0x7f46340cd6d0 0x7f463406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 -- 172.21.15.3:0/3684095336 >> 172.21.15.3:0/3684095336 conn(0x7f463406ab20 msgr2=0x7f463406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 -- 172.21.15.3:0/3684095336 shutdown_connections 2023-11-04T05:29:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.390+0000 7f463a6fc700 1 -- 172.21.15.3:0/3684095336 wait complete. 2023-11-04T05:29:04.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.391+0000 7f463a6fc700 1 Processor -- start 2023-11-04T05:29:04.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.391+0000 7f463a6fc700 1 -- start start 2023-11-04T05:29:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f463a6fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 0x7f46341f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f463a6fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46340cf230 0x7f46341f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f463a6fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f463a6fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341f42d0 con 0x7f46340cd6d0 2023-11-04T05:29:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f463a6fc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463415b540 con 0x7f46340cf230 2023-11-04T05:29:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f463a6fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463415b820 con 0x7f46340cf610 2023-11-04T05:29:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f4633fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 0x7f46341f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f4633fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 0x7f46341f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52684/0 (socket says 172.21.15.3:52684) 2023-11-04T05:29:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f4633fff700 1 -- 172.21.15.3:0/2241630737 learned_addr learned my addr 172.21.15.3:0/2241630737 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f46337fe700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46340cf230 0x7f46341f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.392+0000 7f4638c99700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4633fff700 1 -- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 msgr2=0x7f4634159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4633fff700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4633fff700 1 -- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46340cf230 msgr2=0x7f46341f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4633fff700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46340cf230 0x7f46341f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4633fff700 1 -- 172.21.15.3:0/2241630737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624009550 con 0x7f46340cd6d0 2023-11-04T05:29:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4633fff700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 0x7f46341f3d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4624000c00 tx=0x7f4624004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624004480 con 0x7f46340cd6d0 2023-11-04T05:29:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46240045e0 con 0x7f46340cd6d0 2023-11-04T05:29:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463415bb00 con 0x7f46340cd6d0 2023-11-04T05:29:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46240038e0 con 0x7f46340cd6d0 2023-11-04T05:29:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.393+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463415c0c0 con 0x7f46340cd6d0 2023-11-04T05:29:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.394+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634074df0 con 0x7f46340cd6d0 2023-11-04T05:29:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.395+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4624021070 con 0x7f46340cd6d0 2023-11-04T05:29:04.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.395+0000 7f4630ff9700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f462006af00 0x7f462006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.395+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f462408aed0 con 0x7f46340cd6d0 2023-11-04T05:29:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.395+0000 7f46337fe700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f462006af00 0x7f462006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.396+0000 7f46337fe700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f462006af00 0x7f462006d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f463415b0b0 tx=0x7f4628008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.400+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4624055c50 con 0x7f46340cd6d0 2023-11-04T05:29:04.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.650+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46340b3e80 con 0x7f46340cd6d0 2023-11-04T05:29:04.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.650+0000 7f4630ff9700 1 -- 172.21.15.3:0/2241630737 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4624029070 con 0x7f46340cd6d0 2023-11-04T05:29:04.653 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-04T05:29:04.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f462006af00 msgr2=0x7f462006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:04.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f462006af00 0x7f462006d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f463415b0b0 tx=0x7f4628008040 comp rx=0 tx=0).stop 2023-11-04T05:29:04.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 msgr2=0x7f46341f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:04.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 0x7f46341f3d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4624000c00 tx=0x7f4624004750 comp rx=0 tx=0).stop 2023-11-04T05:29:04.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 shutdown_connections 2023-11-04T05:29:04.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f462006af00 0x7f462006d3c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46340cf610 0x7f4634159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46340cd6d0 0x7f46341f3d90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 --2- 172.21.15.3:0/2241630737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46340cf230 0x7f46341f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 >> 172.21.15.3:0/2241630737 conn(0x7f463406ab20 msgr2=0x7f463406c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:04.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 shutdown_connections 2023-11-04T05:29:04.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:04.653+0000 7f463a6fc700 1 -- 172.21.15.3:0/2241630737 wait complete. 2023-11-04T05:29:04.667 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-04T05:29:04.667 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-04T05:29:04.667 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-04T05:29:04.668 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-04T05:29:04.668 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-04T05:29:04.668 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-04T05:29:04.668 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-04T05:29:04.669 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-04T05:29:04.669 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-04T05:29:04.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.670 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-04T05:29:04.687 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:04.688 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:04.688 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:04.688 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:04.688 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:04.689 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:04.689 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:04.689 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:04.689 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:04.689 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:04.690 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:04.690 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:04.690 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-04T05:29:04.690 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:04.690 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-04T05:29:04.691 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42363sec preferred_lft 42363sec 2023-11-04T05:29:04.691 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::6c9a:ab52:aed8:81ea/64 scope link noprefixroute 2023-11-04T05:29:04.691 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:04.692 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:04.692 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 32:6f:88:09:4e:eb brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:04.692 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-04T05:29:04.692 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:04.692 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::38b1:b4ff:fe24:fece/64 scope link 2023-11-04T05:29:04.693 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:04.693 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-04T05:29:04.693 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 32:6f:88:09:4e:eb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-04T05:29:04.693 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::306f:88ff:fe09:4eeb/64 scope link tentative 2023-11-04T05:29:04.693 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:04.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.694 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-04T05:29:04.753 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-04T05:29:04.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.755 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-04T05:29:04.814 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-04T05:29:04.814 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-04T05:29:04.814 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-04T05:29:04.815 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-04T05:29:04.815 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-04T05:29:04.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.816 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:04.816 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:04.816 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-04T05:29:04.816 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-04T05:29:04.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-04T05:29:04.817 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:04.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.920 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-04T05:29:04.936 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-04T05:29:04.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.937 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2023-11-04T05:29:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-04T05:29:04.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:04.994 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:04.995 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:04.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-04T05:29:04.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-04T05:29:04.995 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:05.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-04T05:29:05.125 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:05.126 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:05.126 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:05.126 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.126 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:05.127 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.127 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:05.127 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether a6:b3:65:ac:0e:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-04T05:29:05.127 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-04T05:29:05.127 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.128 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::a4b3:65ff:feac:e78/64 scope link tentative 2023-11-04T05:29:05.128 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-04T05:29:05.231 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:05.231 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:05.231 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:05.231 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.232 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:05.232 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.232 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:05.232 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether a6:b3:65:ac:0e:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-04T05:29:05.232 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-04T05:29:05.233 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.233 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::a4b3:65ff:feac:e78/64 scope link tentative 2023-11-04T05:29:05.233 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:05.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-04T05:29:05.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.244 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:05.244 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:05.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-04T05:29:05.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-04T05:29:05.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-04T05:29:05.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-04T05:29:05.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-04T05:29:05.246 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:05.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.457 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-04T05:29:05.457 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-04T05:29:05.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2023-11-04T05:29:05.458 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2023-11-04T05:29:05.458 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-04T05:29:05.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:05.569 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-04T05:29:05.591 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-04T05:29:05.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:05.592 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-04T05:29:05.647 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-04T05:29:05.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.648 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-04T05:29:05.732 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-04T05:29:05.733 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-04T05:29:05.733 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2023-11-04T05:29:05.742 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-04T05:29:05.742 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-04T05:29:05.742 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2023-11-04T05:29:05.742 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:05.743 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-04T05:29:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-04T05:29:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2023-11-04T05:29:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:05.768 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:05.768 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790" 2023-11-04T05:29:05.768 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2023-11-04T05:29:05.768 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb" will pass to kernel 2023-11-04T05:29:05.784 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-04T05:29:05.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.785 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-04T05:29:05.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.811 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:05.884 INFO:teuthology.orchestra.run.smithi188.stdout:client4586 2023-11-04T05:29:05.885 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:05.886 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:05.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:05.942 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:06.015 INFO:teuthology.orchestra.run.smithi188.stdout:client4586 2023-11-04T05:29:06.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:06.016 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:06.088 INFO:teuthology.orchestra.run.smithi188.stdout:client4586 2023-11-04T05:29:06.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:06.088 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7a64e299-05b0-4fd0-819f-2262a06846bb.client4586/status') 2023-11-04T05:29:06.167 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4586 (3)192.168.0.2:0/2092177759 2023-11-04T05:29:06.167 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2023-11-04T05:29:06.168 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2023-11-04T05:29:06.169 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2023-11-04T05:29:06.169 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 2.1313e-05 s, 3.2 MB/s 2023-11-04T05:29:06.169 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-04T05:29:06.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.340+0000 7f449adbb700 1 Processor -- start 2023-11-04T05:29:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.340+0000 7f449adbb700 1 -- start start 2023-11-04T05:29:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f449adbb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4494152aa0 0x7f44940cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f449adbb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f449415bd70 0x7f44940ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f449adbb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44940b75f0 con 0x7f44941526c0 2023-11-04T05:29:06.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f449adbb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449415cdd0 con 0x7f4494152aa0 2023-11-04T05:29:06.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f449adbb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44940ce8a0 con 0x7f449415bd70 2023-11-04T05:29:06.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4498b57700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44941526c0 0x7f44940b6ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58044/0 (socket says 172.21.15.3:58044) 2023-11-04T05:29:06.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4498b57700 1 -- 172.21.15.3:0/315078127 learned_addr learned my addr 172.21.15.3:0/315078127 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:06.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4493fff700 1 --2- 172.21.15.3:0/315078127 >> v2:172.21.15.188:3300/0 conn(0x7f4494152aa0 0x7f44940cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4499358700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f449415bd70 0x7f44940ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f44937fe700 1 -- 172.21.15.3:0/315078127 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779838836 0 0) 0x7f44940b75f0 con 0x7f44941526c0 2023-11-04T05:29:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f44937fe700 1 -- 172.21.15.3:0/315078127 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840035b0 con 0x7f44941526c0 2023-11-04T05:29:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4493fff700 1 -- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f449415bd70 msgr2=0x7f44940ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4493fff700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f449415bd70 0x7f44940ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4493fff700 1 -- 172.21.15.3:0/315078127 >> v1:172.21.15.3:6789/0 conn(0x7f44941526c0 legacy=0x7f44940b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4493fff700 1 -- 172.21.15.3:0/315078127 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449415c250 con 0x7f4494152aa0 2023-11-04T05:29:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.341+0000 7f4499358700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f449415bd70 0x7f44940ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f4493fff700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4494152aa0 0x7f44940cde20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4480009840 tx=0x7f4480009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efda8f17ac6a2b75 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f44937fe700 1 -- 172.21.15.3:0/315078127 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480011830 con 0x7f4494152aa0 2023-11-04T05:29:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f44937fe700 1 -- 172.21.15.3:0/315078127 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4480011990 con 0x7f4494152aa0 2023-11-04T05:29:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f44937fe700 1 -- 172.21.15.3:0/315078127 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480011d20 con 0x7f4494152aa0 2023-11-04T05:29:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 -- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4494152aa0 msgr2=0x7f44940cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4494152aa0 0x7f44940cde20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4480009840 tx=0x7f4480009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 -- 172.21.15.3:0/315078127 shutdown_connections 2023-11-04T05:29:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f449415bd70 0x7f44940ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 --2- 172.21.15.3:0/315078127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4494152aa0 0x7f44940cde20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 -- 172.21.15.3:0/315078127 >> 172.21.15.3:0/315078127 conn(0x7f449406ab20 msgr2=0x7f449406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 -- 172.21.15.3:0/315078127 shutdown_connections 2023-11-04T05:29:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.342+0000 7f449adbb700 1 -- 172.21.15.3:0/315078127 wait complete. 2023-11-04T05:29:06.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.344+0000 7f449adbb700 1 Processor -- start 2023-11-04T05:29:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.344+0000 7f449adbb700 1 -- start start 2023-11-04T05:29:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f449adbb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44941526c0 0x7f44941f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f449adbb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 0x7f44940c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f449adbb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44941473e0 0x7f449415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f449adbb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449415dec0 con 0x7f449415bd70 2023-11-04T05:29:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f449adbb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449415e1a0 con 0x7f44941526c0 2023-11-04T05:29:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f449adbb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449415e480 con 0x7f44941473e0 2023-11-04T05:29:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 0x7f44940c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 0x7f44940c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52704/0 (socket says 172.21.15.3:52704) 2023-11-04T05:29:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 -- 172.21.15.3:0/2349749083 learned_addr learned my addr 172.21.15.3:0/2349749083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4499358700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44941473e0 0x7f449415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4498b57700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44941526c0 0x7f44941f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 -- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44941473e0 msgr2=0x7f449415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44941473e0 0x7f449415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 -- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44941526c0 msgr2=0x7f44941f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44941526c0 0x7f44941f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 -- 172.21.15.3:0/2349749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c009a70 con 0x7f449415bd70 2023-11-04T05:29:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4498b57700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44941526c0 0x7f44941f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4499358700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44941473e0 0x7f449415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f4493fff700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 0x7f44940c50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44800090b0 tx=0x7f4480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.345+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480004590 con 0x7f449415bd70 2023-11-04T05:29:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.346+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448001f070 con 0x7f449415bd70 2023-11-04T05:29:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.346+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800038e0 con 0x7f449415bd70 2023-11-04T05:29:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.346+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44800094a0 con 0x7f449415bd70 2023-11-04T05:29:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.346+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449415db90 con 0x7f449415bd70 2023-11-04T05:29:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.346+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449415e000 con 0x7f449415bd70 2023-11-04T05:29:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.347+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4480003a40 con 0x7f449415bd70 2023-11-04T05:29:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.347+0000 7f44917fa700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f447806af00 0x7f447806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.347+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4480021070 con 0x7f449415bd70 2023-11-04T05:29:06.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.348+0000 7f4498b57700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f447806af00 0x7f447806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.348+0000 7f4498b57700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f447806af00 0x7f447806d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4488007fd0 tx=0x7f4488007dc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:06.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.352+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f449415e000 con 0x7f449415bd70 2023-11-04T05:29:06.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.555+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f449415e000 con 0x7f449415bd70 2023-11-04T05:29:06.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.557+0000 7f44917fa700 1 -- 172.21.15.3:0/2349749083 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7f449415e000 con 0x7f449415bd70 2023-11-04T05:29:06.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:06.559 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:06.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f447806af00 msgr2=0x7f447806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f447806af00 0x7f447806d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4488007fd0 tx=0x7f4488007dc0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 msgr2=0x7f44940c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 0x7f44940c50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44800090b0 tx=0x7f4480004750 comp rx=0 tx=0).stop 2023-11-04T05:29:06.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 shutdown_connections 2023-11-04T05:29:06.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f447806af00 0x7f447806d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44941526c0 0x7f44941f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44941473e0 0x7f449415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 --2- 172.21.15.3:0/2349749083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f449415bd70 0x7f44940c50c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 >> 172.21.15.3:0/2349749083 conn(0x7f449406ab20 msgr2=0x7f44940b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:06.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 shutdown_connections 2023-11-04T05:29:06.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.559+0000 7f449adbb700 1 -- 172.21.15.3:0/2349749083 wait complete. 2023-11-04T05:29:06.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:06.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.700+0000 7f2b2359e700 1 Processor -- start 2023-11-04T05:29:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.700+0000 7f2b2359e700 1 -- start start 2023-11-04T05:29:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b240cd7c0 0x7f2b24161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2359e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b2415db60 0x7f2b240b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b24161540 con 0x7f2b2415d780 2023-11-04T05:29:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b240ce720 con 0x7f2b240cd7c0 2023-11-04T05:29:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2359e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240ce9b0 con 0x7f2b2415db60 2023-11-04T05:29:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b21d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b240cd7c0 0x7f2b24161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2b2415d780 0x7f2b240ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58058/0 (socket says 172.21.15.3:58058) 2023-11-04T05:29:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b2259c700 1 -- 172.21.15.3:0/3653416832 learned_addr learned my addr 172.21.15.3:0/3653416832 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b22d9d700 1 --2- 172.21.15.3:0/3653416832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b2415db60 0x7f2b240b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b21d9b700 1 -- 172.21.15.3:0/3653416832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b2415db60 msgr2=0x7f2b240b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b21d9b700 1 --2- 172.21.15.3:0/3653416832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b2415db60 0x7f2b240b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.701+0000 7f2b21d9b700 1 -- 172.21.15.3:0/3653416832 >> v1:172.21.15.3:6789/0 conn(0x7f2b2415d780 legacy=0x7f2b240ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b21d9b700 1 -- 172.21.15.3:0/3653416832 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b241617b0 con 0x7f2b240cd7c0 2023-11-04T05:29:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2159a700 1 -- 172.21.15.3:0/3653416832 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830377788 0 0) 0x7f2b24161540 con 0x7f2b2415d780 2023-11-04T05:29:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b21d9b700 1 --2- 172.21.15.3:0/3653416832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b240cd7c0 0x7f2b24161fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009840 tx=0x7f2b14009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef748a5e5a822293 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2159a700 1 -- 172.21.15.3:0/3653416832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011830 con 0x7f2b240cd7c0 2023-11-04T05:29:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2159a700 1 -- 172.21.15.3:0/3653416832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14011990 con 0x7f2b240cd7c0 2023-11-04T05:29:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2159a700 1 -- 172.21.15.3:0/3653416832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011d20 con 0x7f2b240cd7c0 2023-11-04T05:29:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 -- 172.21.15.3:0/3653416832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b240cd7c0 msgr2=0x7f2b24161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 --2- 172.21.15.3:0/3653416832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b240cd7c0 0x7f2b24161fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009840 tx=0x7f2b14009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 -- 172.21.15.3:0/3653416832 shutdown_connections 2023-11-04T05:29:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 --2- 172.21.15.3:0/3653416832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b2415db60 0x7f2b240b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 --2- 172.21.15.3:0/3653416832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b240cd7c0 0x7f2b24161fb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 -- 172.21.15.3:0/3653416832 >> 172.21.15.3:0/3653416832 conn(0x7f2b2406e200 msgr2=0x7f2b2406e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:06.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.702+0000 7f2b2359e700 1 -- 172.21.15.3:0/3653416832 shutdown_connections 2023-11-04T05:29:06.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.703+0000 7f2b2359e700 1 -- 172.21.15.3:0/3653416832 wait complete. 2023-11-04T05:29:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.704+0000 7f2b2359e700 1 Processor -- start 2023-11-04T05:29:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.704+0000 7f2b2359e700 1 -- start start 2023-11-04T05:29:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b240cd7c0 0x7f2b240b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2415d780 0x7f2b240ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 0x7f2b240ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240ae040 con 0x7f2b2415db60 2023-11-04T05:29:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240c56b0 con 0x7f2b2415d780 2023-11-04T05:29:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240ad430 con 0x7f2b240cd7c0 2023-11-04T05:29:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 0x7f2b240ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 0x7f2b240ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52720/0 (socket says 172.21.15.3:52720) 2023-11-04T05:29:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 -- 172.21.15.3:0/4128506497 learned_addr learned my addr 172.21.15.3:0/4128506497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2259c700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b240cd7c0 0x7f2b240b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b21d9b700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2415d780 0x7f2b240ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 -- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b240cd7c0 msgr2=0x7f2b240b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b240cd7c0 0x7f2b240b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 -- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2415d780 msgr2=0x7f2b240ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2415d780 0x7f2b240ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 -- 172.21.15.3:0/4128506497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140094a0 con 0x7f2b2415db60 2023-11-04T05:29:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b22d9d700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 0x7f2b240ace20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00efc0 tx=0x7f2b1c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c009b20 con 0x7f2b2415db60 2023-11-04T05:29:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00f040 con 0x7f2b2415db60 2023-11-04T05:29:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b240ad710 con 0x7f2b2415db60 2023-11-04T05:29:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00cc90 con 0x7f2b2415db60 2023-11-04T05:29:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.705+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b240adcd0 con 0x7f2b2415db60 2023-11-04T05:29:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.706+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2404f730 con 0x7f2b2415db60 2023-11-04T05:29:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.707+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b1c014480 con 0x7f2b2415db60 2023-11-04T05:29:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.707+0000 7f2b12ffd700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b0406af00 0x7f2b0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.708+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b1c08a320 con 0x7f2b2415db60 2023-11-04T05:29:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.708+0000 7f2b2259c700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b0406af00 0x7f2b0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.708+0000 7f2b2259c700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b0406af00 0x7f2b0406d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007fd0 tx=0x7f2b18007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.712+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2b1c0550a0 con 0x7f2b2415db60 2023-11-04T05:29:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.912+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b240b63c0 con 0x7f2b2415db60 2023-11-04T05:29:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.913+0000 7f2b12ffd700 1 -- 172.21.15.3:0/4128506497 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b1c0586c0 con 0x7f2b2415db60 2023-11-04T05:29:06.915 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b0406af00 msgr2=0x7f2b0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b0406af00 0x7f2b0406d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007fd0 tx=0x7f2b18007de0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 msgr2=0x7f2b240ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 0x7f2b240ace20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00efc0 tx=0x7f2b1c009f60 comp rx=0 tx=0).stop 2023-11-04T05:29:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 shutdown_connections 2023-11-04T05:29:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b0406af00 0x7f2b0406d3c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b2415db60 0x7f2b240ace20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b240cd7c0 0x7f2b240b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 --2- 172.21.15.3:0/4128506497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2415d780 0x7f2b240ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 >> 172.21.15.3:0/4128506497 conn(0x7f2b2406e200 msgr2=0x7f2b240b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 shutdown_connections 2023-11-04T05:29:06.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:06.915+0000 7f2b2359e700 1 -- 172.21.15.3:0/4128506497 wait complete. 2023-11-04T05:29:06.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.056+0000 7f18715ea700 1 Processor -- start 2023-11-04T05:29:07.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.056+0000 7f18715ea700 1 -- start start 2023-11-04T05:29:07.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f18715ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c15ba10 0x7f186c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f18715ea700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f186c0d0d70 0x7f186c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f18715ea700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f186c0d11f0 0x7f186c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f18715ea700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c070a90 con 0x7f186c0d0d70 2023-11-04T05:29:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f18715ea700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c0718f0 con 0x7f186c0d11f0 2023-11-04T05:29:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f18715ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0c5720 con 0x7f186c15ba10 2023-11-04T05:29:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f186c0d0d70 0x7f186c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f186c0d0d70 0x7f186c0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52724/0 (socket says 172.21.15.3:52724) 2023-11-04T05:29:07.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 -- 172.21.15.3:0/1385762990 learned_addr learned my addr 172.21.15.3:0/1385762990 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:07.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186b7fe700 1 --2- 172.21.15.3:0/1385762990 >> v2:172.21.15.188:3300/0 conn(0x7f186c0d11f0 0x7f186c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186affd700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c15ba10 0x7f186c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 -- 172.21.15.3:0/1385762990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c15ba10 msgr2=0x7f186c0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:07.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c15ba10 0x7f186c0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 -- 172.21.15.3:0/1385762990 >> v2:172.21.15.188:3300/0 conn(0x7f186c0d11f0 msgr2=0x7f186c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 --2- 172.21.15.3:0/1385762990 >> v2:172.21.15.188:3300/0 conn(0x7f186c0d11f0 0x7f186c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186a7fc700 1 -- 172.21.15.3:0/1385762990 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c070ce0 con 0x7f186c0d0d70 2023-11-04T05:29:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.057+0000 7f186affd700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c15ba10 0x7f186c0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:07.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f186a7fc700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0d0d70 0x7f186c0b3110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f186c0c5160 tx=0x7f185c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e021ceed96ad1131 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f1869ffb700 1 -- 172.21.15.3:0/1385762990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00b870 con 0x7f186c0d0d70 2023-11-04T05:29:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f1869ffb700 1 -- 172.21.15.3:0/1385762990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c00b9d0 con 0x7f186c0d0d70 2023-11-04T05:29:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f1869ffb700 1 -- 172.21.15.3:0/1385762990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00bd60 con 0x7f186c0d0d70 2023-11-04T05:29:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 -- 172.21.15.3:0/1385762990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0d0d70 msgr2=0x7f186c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0d0d70 0x7f186c0b3110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f186c0c5160 tx=0x7f185c009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 -- 172.21.15.3:0/1385762990 shutdown_connections 2023-11-04T05:29:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 --2- 172.21.15.3:0/1385762990 >> v2:172.21.15.188:3300/0 conn(0x7f186c0d11f0 0x7f186c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c15ba10 0x7f186c0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 --2- 172.21.15.3:0/1385762990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0d0d70 0x7f186c0b3110 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 -- 172.21.15.3:0/1385762990 >> 172.21.15.3:0/1385762990 conn(0x7f186c0ce110 msgr2=0x7f186c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 -- 172.21.15.3:0/1385762990 shutdown_connections 2023-11-04T05:29:07.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.058+0000 7f18715ea700 1 -- 172.21.15.3:0/1385762990 wait complete. 2023-11-04T05:29:07.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 Processor -- start 2023-11-04T05:29:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 -- start start 2023-11-04T05:29:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0d0d70 0x7f186c0b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c15ba10 0x7f186c0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 0x7f186c14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0b19f0 con 0x7f186c0b1570 2023-11-04T05:29:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c14d6c0 con 0x7f186c15ba10 2023-11-04T05:29:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.060+0000 7f18715ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0b7300 con 0x7f186c0d0d70 2023-11-04T05:29:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 0x7f186c14c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 0x7f186c14c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52732/0 (socket says 172.21.15.3:52732) 2023-11-04T05:29:07.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 -- 172.21.15.3:0/2075388546 learned_addr learned my addr 172.21.15.3:0/2075388546 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186a7fc700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c15ba10 0x7f186c0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186affd700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0d0d70 0x7f186c0b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 -- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0d0d70 msgr2=0x7f186c0b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0d0d70 0x7f186c0b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 -- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c15ba10 msgr2=0x7f186c0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c15ba10 0x7f186c0b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 -- 172.21.15.3:0/2075388546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009550 con 0x7f186c0b1570 2023-11-04T05:29:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186a7fc700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c15ba10 0x7f186c0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186affd700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0d0d70 0x7f186c0b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f186b7fe700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 0x7f186c14c970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1864007d90 tx=0x7f186400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400ede0 con 0x7f186c0b1570 2023-11-04T05:29:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864014420 con 0x7f186c0b1570 2023-11-04T05:29:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c14a850 con 0x7f186c0b1570 2023-11-04T05:29:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640147d0 con 0x7f186c0b1570 2023-11-04T05:29:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.061+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c14ae10 con 0x7f186c0b1570 2023-11-04T05:29:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.062+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c074df0 con 0x7f186c0b1570 2023-11-04T05:29:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.063+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1864014930 con 0x7f186c0b1570 2023-11-04T05:29:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.063+0000 7f18537fe700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f185806af00 0x7f185806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.063+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f186408a130 con 0x7f186c0b1570 2023-11-04T05:29:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.064+0000 7f186affd700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f185806af00 0x7f185806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.064+0000 7f186affd700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f185806af00 0x7f185806d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18600089b0 tx=0x7f1860008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.068+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1864054e10 con 0x7f186c0b1570 2023-11-04T05:29:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.270+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f186c0cee70 con 0x7f186c0b1570 2023-11-04T05:29:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.270+0000 7f18537fe700 1 -- 172.21.15.3:0/2075388546 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1864058430 con 0x7f186c0b1570 2023-11-04T05:29:07.273 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:07.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.272+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f185806af00 msgr2=0x7f185806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.272+0000 7f18715ea700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f185806af00 0x7f185806d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18600089b0 tx=0x7f1860008040 comp rx=0 tx=0).stop 2023-11-04T05:29:07.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 msgr2=0x7f186c14c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 0x7f186c14c970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1864007d90 tx=0x7f186400e3c0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 shutdown_connections 2023-11-04T05:29:07.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f185806af00 0x7f185806d3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f186c0b1570 0x7f186c14c970 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0d0d70 0x7f186c0b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 --2- 172.21.15.3:0/2075388546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c15ba10 0x7f186c0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 >> 172.21.15.3:0/2075388546 conn(0x7f186c0ce110 msgr2=0x7f186c0cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 shutdown_connections 2023-11-04T05:29:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.273+0000 7f18715ea700 1 -- 172.21.15.3:0/2075388546 wait complete. 2023-11-04T05:29:07.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.412+0000 7f78bbc9e700 1 Processor -- start 2023-11-04T05:29:07.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.413+0000 7f78bbc9e700 1 -- start start 2023-11-04T05:29:07.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.413+0000 7f78bbc9e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78b40ceaa0 0x7f78b40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.413+0000 7f78bbc9e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78b415b430 0x7f78b40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.413+0000 7f78bbc9e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b415b870 con 0x7f78b40cee80 2023-11-04T05:29:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78bbc9e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40ccc70 con 0x7f78b40ceaa0 2023-11-04T05:29:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78bbc9e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40cdee0 con 0x7f78b415b430 2023-11-04T05:29:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78b40ceaa0 0x7f78b40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78b40ceaa0 0x7f78b40cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52736/0 (socket says 172.21.15.3:52736) 2023-11-04T05:29:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 -- 172.21.15.3:0/4263342259 learned_addr learned my addr 172.21.15.3:0/4263342259 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78ba23b700 1 --2- 172.21.15.3:0/4263342259 >> v2:172.21.15.188:3300/0 conn(0x7f78b415b430 0x7f78b40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 -- 172.21.15.3:0/4263342259 >> v2:172.21.15.188:3300/0 conn(0x7f78b415b430 msgr2=0x7f78b40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 --2- 172.21.15.3:0/4263342259 >> v2:172.21.15.188:3300/0 conn(0x7f78b415b430 0x7f78b40caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 -- 172.21.15.3:0/4263342259 >> v1:172.21.15.188:6789/0 conn(0x7f78b40cee80 legacy=0x7f78b415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 -- 172.21.15.3:0/4263342259 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b415baa0 con 0x7f78b40ceaa0 2023-11-04T05:29:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78ba23b700 1 --2- 172.21.15.3:0/4263342259 >> v2:172.21.15.188:3300/0 conn(0x7f78b415b430 0x7f78b40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b9a3a700 1 --2- 172.21.15.3:0/4263342259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b40ceaa0 0x7f78b40cd9a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78b40cac20 tx=0x7f78a800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8365b3e708ec11 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b8a38700 1 -- 172.21.15.3:0/4263342259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800dd30 con 0x7f78b40ceaa0 2023-11-04T05:29:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b8a38700 1 -- 172.21.15.3:0/4263342259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a800de90 con 0x7f78b40ceaa0 2023-11-04T05:29:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.414+0000 7f78b8a38700 1 -- 172.21.15.3:0/4263342259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8010690 con 0x7f78b40ceaa0 2023-11-04T05:29:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 -- 172.21.15.3:0/4263342259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b40ceaa0 msgr2=0x7f78b40cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/4263342259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b40ceaa0 0x7f78b40cd9a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78b40cac20 tx=0x7f78a800be70 comp rx=0 tx=0).stop 2023-11-04T05:29:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 -- 172.21.15.3:0/4263342259 shutdown_connections 2023-11-04T05:29:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/4263342259 >> v2:172.21.15.188:3300/0 conn(0x7f78b415b430 0x7f78b40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/4263342259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b40ceaa0 0x7f78b40cd9a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 -- 172.21.15.3:0/4263342259 >> 172.21.15.3:0/4263342259 conn(0x7f78b4072560 msgr2=0x7f78b4099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 -- 172.21.15.3:0/4263342259 shutdown_connections 2023-11-04T05:29:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.415+0000 7f78bbc9e700 1 -- 172.21.15.3:0/4263342259 wait complete. 2023-11-04T05:29:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.416+0000 7f78bbc9e700 1 Processor -- start 2023-11-04T05:29:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.416+0000 7f78bbc9e700 1 -- start start 2023-11-04T05:29:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78bbc9e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b40ceaa0 0x7f78b40c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78bbc9e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40cee80 0x7f78b41e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78bbc9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 0x7f78b41eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78bbc9e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b41592f0 con 0x7f78b415b430 2023-11-04T05:29:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78bbc9e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4158670 con 0x7f78b40ceaa0 2023-11-04T05:29:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78bbc9e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4158950 con 0x7f78b40cee80 2023-11-04T05:29:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 0x7f78b41eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 0x7f78b41eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52746/0 (socket says 172.21.15.3:52746) 2023-11-04T05:29:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 -- 172.21.15.3:0/2468984805 learned_addr learned my addr 172.21.15.3:0/2468984805 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78b9239700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40cee80 0x7f78b41e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78b9a3a700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b40ceaa0 0x7f78b40c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 -- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40cee80 msgr2=0x7f78b41e56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40cee80 0x7f78b41e56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 -- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b40ceaa0 msgr2=0x7f78b40c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b40ceaa0 0x7f78b40c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78ba23b700 1 -- 172.21.15.3:0/2468984805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a800b7f0 con 0x7f78b415b430 2023-11-04T05:29:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78b9239700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40cee80 0x7f78b41e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.417+0000 7f78b9a3a700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b40ceaa0 0x7f78b40c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.418+0000 7f78ba23b700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 0x7f78b41eff20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f78ac007ef0 tx=0x7f78ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.418+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00de60 con 0x7f78b415b430 2023-11-04T05:29:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.418+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00f040 con 0x7f78b415b430 2023-11-04T05:29:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.418+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b4158c30 con 0x7f78b415b430 2023-11-04T05:29:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.418+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003c60 con 0x7f78b415b430 2023-11-04T05:29:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.418+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b41f30b0 con 0x7f78b415b430 2023-11-04T05:29:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.419+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4074df0 con 0x7f78b415b430 2023-11-04T05:29:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.419+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78ac004030 con 0x7f78b415b430 2023-11-04T05:29:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.420+0000 7f78a67fc700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f789806af00 0x7f789806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.420+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f78ac089ba0 con 0x7f78b415b430 2023-11-04T05:29:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.420+0000 7f78b9a3a700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f789806af00 0x7f789806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.420+0000 7f78b9a3a700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f789806af00 0x7f789806d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78a800beb0 tx=0x7f78a8012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.424+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f78ac054880 con 0x7f78b415b430 2023-11-04T05:29:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.625+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78b40cd940 con 0x7f78b415b430 2023-11-04T05:29:07.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.625+0000 7f78a67fc700 1 -- 172.21.15.3:0/2468984805 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78ac01a0a0 con 0x7f78b415b430 2023-11-04T05:29:07.628 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:07.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f789806af00 msgr2=0x7f789806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f789806af00 0x7f789806d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78a800beb0 tx=0x7f78a8012000 comp rx=0 tx=0).stop 2023-11-04T05:29:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 msgr2=0x7f78b41eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 0x7f78b41eff20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f78ac007ef0 tx=0x7f78ac00d3b0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 shutdown_connections 2023-11-04T05:29:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f789806af00 0x7f789806d3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b40ceaa0 0x7f78b40c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b415b430 0x7f78b41eff20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 --2- 172.21.15.3:0/2468984805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40cee80 0x7f78b41e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 >> 172.21.15.3:0/2468984805 conn(0x7f78b4072560 msgr2=0x7f78b40736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:07.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 shutdown_connections 2023-11-04T05:29:07.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.627+0000 7f78bbc9e700 1 -- 172.21.15.3:0/2468984805 wait complete. 2023-11-04T05:29:07.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.768+0000 7f9c37b2b700 1 Processor -- start 2023-11-04T05:29:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.768+0000 7f9c37b2b700 1 -- start start 2023-11-04T05:29:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c37b2b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c300ceaa0 0x7f9c300cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c37b2b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c300cee80 0x7f9c3015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c37b2b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c300ccc70 con 0x7f9c3015b260 2023-11-04T05:29:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c37b2b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300cdee0 con 0x7f9c300cee80 2023-11-04T05:29:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c37b2b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3015baa0 con 0x7f9c300ceaa0 2023-11-04T05:29:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c300cee80 0x7f9c3015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c300cee80 0x7f9c3015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52752/0 (socket says 172.21.15.3:52752) 2023-11-04T05:29:07.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 -- 172.21.15.3:0/2302088096 learned_addr learned my addr 172.21.15.3:0/2302088096 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:07.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c360c8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c3015b260 0x7f9c300caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58102/0 (socket says 172.21.15.3:58102) 2023-11-04T05:29:07.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c358c7700 1 --2- 172.21.15.3:0/2302088096 >> v2:172.21.15.188:3300/0 conn(0x7f9c300ceaa0 0x7f9c300cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 -- 172.21.15.3:0/2302088096 >> v2:172.21.15.188:3300/0 conn(0x7f9c300ceaa0 msgr2=0x7f9c300cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 --2- 172.21.15.3:0/2302088096 >> v2:172.21.15.188:3300/0 conn(0x7f9c300ceaa0 0x7f9c300cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 -- 172.21.15.3:0/2302088096 >> v1:172.21.15.3:6789/0 conn(0x7f9c3015b260 legacy=0x7f9c300caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c350c6700 1 -- 172.21.15.3:0/2302088096 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300ccec0 con 0x7f9c300cee80 2023-11-04T05:29:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.769+0000 7f9c348c5700 1 -- 172.21.15.3:0/2302088096 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653552173 0 0) 0x7f9c300ccc70 con 0x7f9c3015b260 2023-11-04T05:29:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c350c6700 1 --2- 172.21.15.3:0/2302088096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c3015ad20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c30072c40 tx=0x7f9c2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf46e50d31b741dc server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c348c5700 1 -- 172.21.15.3:0/2302088096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00b870 con 0x7f9c300cee80 2023-11-04T05:29:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c348c5700 1 -- 172.21.15.3:0/2302088096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c00b9d0 con 0x7f9c300cee80 2023-11-04T05:29:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c348c5700 1 -- 172.21.15.3:0/2302088096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00bd60 con 0x7f9c300cee80 2023-11-04T05:29:07.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2302088096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 msgr2=0x7f9c3015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2302088096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c3015ad20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c30072c40 tx=0x7f9c2c009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2302088096 shutdown_connections 2023-11-04T05:29:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2302088096 >> v2:172.21.15.188:3300/0 conn(0x7f9c300ceaa0 0x7f9c300cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2302088096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c3015ad20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2302088096 >> 172.21.15.3:0/2302088096 conn(0x7f9c30072560 msgr2=0x7f9c30099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2302088096 shutdown_connections 2023-11-04T05:29:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.770+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2302088096 wait complete. 2023-11-04T05:29:07.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.772+0000 7f9c37b2b700 1 Processor -- start 2023-11-04T05:29:07.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.772+0000 7f9c37b2b700 1 -- start start 2023-11-04T05:29:07.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c37b2b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c300ceaa0 0x7f9c300ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c37b2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c30155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c37b2b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3015b260 0x7f9c300b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c37b2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300c54c0 con 0x7f9c300cee80 2023-11-04T05:29:07.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c37b2b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c30155790 con 0x7f9c300ceaa0 2023-11-04T05:29:07.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c37b2b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300c4f00 con 0x7f9c3015b260 2023-11-04T05:29:07.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c30155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c30155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52758/0 (socket says 172.21.15.3:52758) 2023-11-04T05:29:07.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 -- 172.21.15.3:0/2502571685 learned_addr learned my addr 172.21.15.3:0/2502571685 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:07.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c360c8700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3015b260 0x7f9c300b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c358c7700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c300ceaa0 0x7f9c300ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 -- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3015b260 msgr2=0x7f9c300b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3015b260 0x7f9c300b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 -- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c300ceaa0 msgr2=0x7f9c300ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c300ceaa0 0x7f9c300ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c350c6700 1 -- 172.21.15.3:0/2502571685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009550 con 0x7f9c300cee80 2023-11-04T05:29:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.773+0000 7f9c358c7700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c300ceaa0 0x7f9c300ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.774+0000 7f9c350c6700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c30155250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009c10 tx=0x7f9c2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.774+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c004310 con 0x7f9c300cee80 2023-11-04T05:29:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.774+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c01f070 con 0x7f9c300cee80 2023-11-04T05:29:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.774+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c300c51e0 con 0x7f9c300cee80 2023-11-04T05:29:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.774+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0036a0 con 0x7f9c300cee80 2023-11-04T05:29:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.774+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c300b2830 con 0x7f9c300cee80 2023-11-04T05:29:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.775+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c300b2010 con 0x7f9c300cee80 2023-11-04T05:29:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.775+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c2c021030 con 0x7f9c300cee80 2023-11-04T05:29:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.775+0000 7f9c267fc700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9c1406af00 0x7f9c1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:07.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.776+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9c2c08ac50 con 0x7f9c300cee80 2023-11-04T05:29:07.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.776+0000 7f9c358c7700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9c1406af00 0x7f9c1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:07.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.776+0000 7f9c358c7700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9c1406af00 0x7f9c1406d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009a00 tx=0x7f9c20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:07.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.780+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f9c300b2010 con 0x7f9c300cee80 2023-11-04T05:29:07.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.982+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9c300b2010 con 0x7f9c300cee80 2023-11-04T05:29:07.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.984+0000 7f9c267fc700 1 -- 172.21.15.3:0/2502571685 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9c300b2010 con 0x7f9c300cee80 2023-11-04T05:29:07.986 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9c1406af00 msgr2=0x7f9c1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9c1406af00 0x7f9c1406d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009a00 tx=0x7f9c20008040 comp rx=0 tx=0).stop 2023-11-04T05:29:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 msgr2=0x7f9c30155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c30155250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009c10 tx=0x7f9c2c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:07.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 shutdown_connections 2023-11-04T05:29:07.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9c1406af00 0x7f9c1406d3c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c300ceaa0 0x7f9c300ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3015b260 0x7f9c300b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 --2- 172.21.15.3:0/2502571685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c300cee80 0x7f9c30155250 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:07.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 >> 172.21.15.3:0/2502571685 conn(0x7f9c30072560 msgr2=0x7f9c300736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:07.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 shutdown_connections 2023-11-04T05:29:07.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:07.986+0000 7f9c37b2b700 1 -- 172.21.15.3:0/2502571685 wait complete. 2023-11-04T05:29:08.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:08.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb936524700 1 Processor -- start 2023-11-04T05:29:08.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb936524700 1 -- start start 2023-11-04T05:29:08.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb936524700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb93015ecc0 0x7fb93015f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb936524700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93015dff0 con 0x7fb93015dc10 2023-11-04T05:29:08.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb936524700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb930071e80 con 0x7fb93006ab20 2023-11-04T05:29:08.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb936524700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb930161e50 con 0x7fb93015ecc0 2023-11-04T05:29:08.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb93015ecc0 0x7fb93015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb93015dc10 0x7fb930161690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58112/0 (socket says 172.21.15.3:58112) 2023-11-04T05:29:08.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92ffff700 1 -- 172.21.15.3:0/1278349665 learned_addr learned my addr 172.21.15.3:0/1278349665 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:08.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb93015ecc0 0x7fb93015f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52766/0 (socket says 172.21.15.3:52766) 2023-11-04T05:29:08.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92f7fe700 1 -- 172.21.15.3:0/1278349665 >> v1:172.21.15.188:6789/0 conn(0x7fb93006ab20 legacy=0x7fb9300725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:08.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92f7fe700 1 -- 172.21.15.3:0/1278349665 >> v1:172.21.15.3:6789/0 conn(0x7fb93015dc10 legacy=0x7fb930161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92f7fe700 1 -- 172.21.15.3:0/1278349665 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930162130 con 0x7fb93015ecc0 2023-11-04T05:29:08.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.125+0000 7fb92effd700 1 -- 172.21.15.3:0/1278349665 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286444036 0 0) 0x7fb918001110 con 0x7fb93015dc10 2023-11-04T05:29:08.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb92f7fe700 1 --2- 172.21.15.3:0/1278349665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb93015f490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb920009840 tx=0x7fb920009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa214e9b38dcc581 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb92effd700 1 -- 172.21.15.3:0/1278349665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920004230 con 0x7fb93015ecc0 2023-11-04T05:29:08.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb92effd700 1 -- 172.21.15.3:0/1278349665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920004390 con 0x7fb93015ecc0 2023-11-04T05:29:08.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb92effd700 1 -- 172.21.15.3:0/1278349665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9200036a0 con 0x7fb93015ecc0 2023-11-04T05:29:08.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 -- 172.21.15.3:0/1278349665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 msgr2=0x7fb93015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 --2- 172.21.15.3:0/1278349665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb93015f490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb920009840 tx=0x7fb920009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:08.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 -- 172.21.15.3:0/1278349665 shutdown_connections 2023-11-04T05:29:08.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 --2- 172.21.15.3:0/1278349665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb93015f490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 -- 172.21.15.3:0/1278349665 >> 172.21.15.3:0/1278349665 conn(0x7fb93006de80 msgr2=0x7fb93006e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:08.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 -- 172.21.15.3:0/1278349665 shutdown_connections 2023-11-04T05:29:08.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.126+0000 7fb936524700 1 -- 172.21.15.3:0/1278349665 wait complete. 2023-11-04T05:29:08.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.127+0000 7fb936524700 1 Processor -- start 2023-11-04T05:29:08.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 -- start start 2023-11-04T05:29:08.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93006ab20 0x7fb9300cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93015dc10 0x7fb9300ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb9300b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301521c0 con 0x7fb93015ecc0 2023-11-04T05:29:08.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9300b4630 con 0x7fb93015dc10 2023-11-04T05:29:08.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb936524700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9300b48d0 con 0x7fb93006ab20 2023-11-04T05:29:08.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.128+0000 7fb934ac1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb9300b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb9300b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52774/0 (socket says 172.21.15.3:52774) 2023-11-04T05:29:08.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 -- 172.21.15.3:0/2552454497 learned_addr learned my addr 172.21.15.3:0/2552454497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:08.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb92f7fe700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93015dc10 0x7fb9300ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb92ffff700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93006ab20 0x7fb9300cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 -- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93006ab20 msgr2=0x7fb9300cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93006ab20 0x7fb9300cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 -- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93015dc10 msgr2=0x7fb9300ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93015dc10 0x7fb9300ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 -- 172.21.15.3:0/2552454497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200094a0 con 0x7fb93015ecc0 2023-11-04T05:29:08.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb934ac1700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb9300b1110 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c7a0 tx=0x7fb92800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928003e10 con 0x7fb93015ecc0 2023-11-04T05:29:08.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb928005070 con 0x7fb93015ecc0 2023-11-04T05:29:08.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9300b4c10 con 0x7fb93015ecc0 2023-11-04T05:29:08.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928005420 con 0x7fb93015ecc0 2023-11-04T05:29:08.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.129+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9300b5450 con 0x7fb93015ecc0 2023-11-04T05:29:08.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.130+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb930074df0 con 0x7fb93015ecc0 2023-11-04T05:29:08.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.131+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb928005580 con 0x7fb93015ecc0 2023-11-04T05:29:08.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.131+0000 7fb92cff9700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb91006af00 0x7fb91006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.131+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb928088ff0 con 0x7fb93015ecc0 2023-11-04T05:29:08.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.131+0000 7fb92ffff700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb91006af00 0x7fb91006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.132+0000 7fb92ffff700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb91006af00 0x7fb91006d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb918000f60 tx=0x7fb918007c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.137+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fb928053c50 con 0x7fb93015ecc0 2023-11-04T05:29:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.338+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9300b2290 con 0x7fb93015ecc0 2023-11-04T05:29:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.339+0000 7fb92cff9700 1 -- 172.21.15.3:0/2552454497 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7fb928017070 con 0x7fb93015ecc0 2023-11-04T05:29:08.341 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:08.342 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb91006af00 msgr2=0x7fb91006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb91006af00 0x7fb91006d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb918000f60 tx=0x7fb918007c20 comp rx=0 tx=0).stop 2023-11-04T05:29:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 msgr2=0x7fb9300b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb9300b1110 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c7a0 tx=0x7fb92800cab0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 shutdown_connections 2023-11-04T05:29:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb91006af00 0x7fb91006d3c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb93015ecc0 0x7fb9300b1110 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93006ab20 0x7fb9300cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 --2- 172.21.15.3:0/2552454497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93015dc10 0x7fb9300ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 >> 172.21.15.3:0/2552454497 conn(0x7fb93006de80 msgr2=0x7fb930160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:08.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 shutdown_connections 2023-11-04T05:29:08.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.341+0000 7fb936524700 1 -- 172.21.15.3:0/2552454497 wait complete. 2023-11-04T05:29:08.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:08.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:08.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 Processor -- start 2023-11-04T05:29:08.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 -- start start 2023-11-04T05:29:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eb81611d0 0x7f5eb80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb806b230 0x7f5eb80d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80caca0 con 0x7f5eb80cd4b0 2023-11-04T05:29:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb80ce0a0 con 0x7f5eb81611d0 2023-11-04T05:29:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.484+0000 7f5ebf231700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80cfe40 con 0x7f5eb806b230 2023-11-04T05:29:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5eb80cd4b0 0x7f5eb806ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58130/0 (socket says 172.21.15.3:58130) 2023-11-04T05:29:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb7fff700 1 -- 172.21.15.3:0/4244439650 learned_addr learned my addr 172.21.15.3:0/4244439650 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebd7ce700 1 --2- 172.21.15.3:0/4244439650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb806b230 0x7f5eb80d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebcfcd700 1 --2- 172.21.15.3:0/4244439650 >> v2:172.21.15.188:3300/0 conn(0x7f5eb81611d0 0x7f5eb80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb77fe700 1 -- 172.21.15.3:0/4244439650 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201637175 0 0) 0x7f5eb80caca0 con 0x7f5eb80cd4b0 2023-11-04T05:29:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb77fe700 1 -- 172.21.15.3:0/4244439650 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea80035b0 con 0x7f5eb80cd4b0 2023-11-04T05:29:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/4244439650 >> v2:172.21.15.188:3300/0 conn(0x7f5eb81611d0 msgr2=0x7f5eb80ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebd7ce700 1 --2- 172.21.15.3:0/4244439650 >> v2:172.21.15.188:3300/0 conn(0x7f5eb81611d0 0x7f5eb80ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/4244439650 >> v1:172.21.15.3:6789/0 conn(0x7f5eb80cd4b0 legacy=0x7f5eb806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/4244439650 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80cf850 con 0x7f5eb806b230 2023-11-04T05:29:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5ebd7ce700 1 --2- 172.21.15.3:0/4244439650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb806b230 0x7f5eb80d02d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009d90 tx=0x7f5eb000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b47df95ea312377 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb77fe700 1 -- 172.21.15.3:0/4244439650 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000e070 con 0x7f5eb806b230 2023-11-04T05:29:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb77fe700 1 -- 172.21.15.3:0/4244439650 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb000bb20 con 0x7f5eb806b230 2023-11-04T05:29:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.485+0000 7f5eb77fe700 1 -- 172.21.15.3:0/4244439650 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0012430 con 0x7f5eb806b230 2023-11-04T05:29:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 -- 172.21.15.3:0/4244439650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb806b230 msgr2=0x7f5eb80d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 --2- 172.21.15.3:0/4244439650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb806b230 0x7f5eb80d02d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009d90 tx=0x7f5eb000b330 comp rx=0 tx=0).stop 2023-11-04T05:29:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 -- 172.21.15.3:0/4244439650 shutdown_connections 2023-11-04T05:29:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 --2- 172.21.15.3:0/4244439650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb806b230 0x7f5eb80d02d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 --2- 172.21.15.3:0/4244439650 >> v2:172.21.15.188:3300/0 conn(0x7f5eb81611d0 0x7f5eb80ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 -- 172.21.15.3:0/4244439650 >> 172.21.15.3:0/4244439650 conn(0x7f5eb8072560 msgr2=0x7f5eb8099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 -- 172.21.15.3:0/4244439650 shutdown_connections 2023-11-04T05:29:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.486+0000 7f5ebf231700 1 -- 172.21.15.3:0/4244439650 wait complete. 2023-11-04T05:29:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.489+0000 7f5ebf231700 1 Processor -- start 2023-11-04T05:29:08.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.489+0000 7f5ebf231700 1 -- start start 2023-11-04T05:29:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb806b230 0x7f5eb815b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80cd4b0 0x7f5eb80b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 0x7f5eb815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb815f0b0 con 0x7f5eb81611d0 2023-11-04T05:29:08.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb81604e0 con 0x7f5eb806b230 2023-11-04T05:29:08.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80b5fd0 con 0x7f5eb80cd4b0 2023-11-04T05:29:08.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 0x7f5eb815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 0x7f5eb815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52788/0 (socket says 172.21.15.3:52788) 2023-11-04T05:29:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/2691232477 learned_addr learned my addr 172.21.15.3:0/2691232477 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5eb7fff700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80cd4b0 0x7f5eb80b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebcfcd700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb806b230 0x7f5eb815b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80cd4b0 msgr2=0x7f5eb80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80cd4b0 0x7f5eb80b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb806b230 msgr2=0x7f5eb815b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb806b230 0x7f5eb815b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 -- 172.21.15.3:0/2691232477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009a70 con 0x7f5eb81611d0 2023-11-04T05:29:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5eb7fff700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80cd4b0 0x7f5eb80b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebcfcd700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb806b230 0x7f5eb815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebd7ce700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 0x7f5eb815fb70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80cff80 tx=0x7f5eb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0012430 con 0x7f5eb81611d0 2023-11-04T05:29:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0020070 con 0x7f5eb81611d0 2023-11-04T05:29:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0004280 con 0x7f5eb81611d0 2023-11-04T05:29:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb815e710 con 0x7f5eb81611d0 2023-11-04T05:29:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.490+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb815ecd0 con 0x7f5eb81611d0 2023-11-04T05:29:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.491+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb815f1f0 con 0x7f5eb81611d0 2023-11-04T05:29:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.491+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5eb00043e0 con 0x7f5eb81611d0 2023-11-04T05:29:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.492+0000 7f5eb57fa700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.492+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5eb0022070 con 0x7f5eb81611d0 2023-11-04T05:29:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.492+0000 7f5ebcfcd700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.492+0000 7f5ebcfcd700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5eb809e6e0 tx=0x7f5eac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.497+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5eb815f1f0 con 0x7f5eb81611d0 2023-11-04T05:29:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.698+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5eb815f1f0 con 0x7f5eb81611d0 2023-11-04T05:29:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.699+0000 7f5eb57fa700 1 -- 172.21.15.3:0/2691232477 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5eb815f1f0 con 0x7f5eb81611d0 2023-11-04T05:29:08.701 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 msgr2=0x7f5e9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5eb809e6e0 tx=0x7f5eac008040 comp rx=0 tx=0).stop 2023-11-04T05:29:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 msgr2=0x7f5eb815fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 0x7f5eb815fb70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80cff80 tx=0x7f5eb0004ad0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 shutdown_connections 2023-11-04T05:29:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5e9c06af00 0x7f5e9c06d3c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb806b230 0x7f5eb815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eb81611d0 0x7f5eb815fb70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 --2- 172.21.15.3:0/2691232477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb80cd4b0 0x7f5eb80b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 >> 172.21.15.3:0/2691232477 conn(0x7f5eb8072560 msgr2=0x7f5eb8153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 shutdown_connections 2023-11-04T05:29:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.702+0000 7f5ebf231700 1 -- 172.21.15.3:0/2691232477 wait complete. 2023-11-04T05:29:08.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:08.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 Processor -- start 2023-11-04T05:29:08.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 -- start start 2023-11-04T05:29:08.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bd40b0f10 0x7f5bd40b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 0x7f5bd40b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd40cd060 con 0x7f5bd40b2820 2023-11-04T05:29:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40b1830 con 0x7f5bd40b0f10 2023-11-04T05:29:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.843+0000 7f5bd9c04700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4072990 con 0x7f5bd40cecc0 2023-11-04T05:29:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 0x7f5bd40b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd37fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bd40b0f10 0x7f5bd40b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 0x7f5bd40b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:43850/0 (socket says 172.21.15.3:43850) 2023-11-04T05:29:08.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/662329039 learned_addr learned my addr 172.21.15.3:0/662329039 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:08.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/662329039 >> v2:172.21.15.188:3300/0 conn(0x7f5bd40b0f10 msgr2=0x7f5bd40b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:08.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 --2- 172.21.15.3:0/662329039 >> v2:172.21.15.188:3300/0 conn(0x7f5bd40b0f10 0x7f5bd40b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/662329039 >> v1:172.21.15.188:6789/0 conn(0x7f5bd40b2820 legacy=0x7f5bd4071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/662329039 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4072c70 con 0x7f5bd40cecc0 2023-11-04T05:29:08.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd2ffd700 1 --2- 172.21.15.3:0/662329039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 0x7f5bd40b22e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40098f0 tx=0x7f5bc4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b617fcc6694b6cda server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd27fc700 1 -- 172.21.15.3:0/662329039 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4011830 con 0x7f5bd40cecc0 2023-11-04T05:29:08.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.844+0000 7f5bd27fc700 1 -- 172.21.15.3:0/662329039 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4011990 con 0x7f5bd40cecc0 2023-11-04T05:29:08.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd27fc700 1 -- 172.21.15.3:0/662329039 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4011d20 con 0x7f5bd40cecc0 2023-11-04T05:29:08.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 -- 172.21.15.3:0/662329039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 msgr2=0x7f5bd40b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/662329039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 0x7f5bd40b22e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40098f0 tx=0x7f5bc4009c00 comp rx=0 tx=0).stop 2023-11-04T05:29:08.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 -- 172.21.15.3:0/662329039 shutdown_connections 2023-11-04T05:29:08.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/662329039 >> v2:172.21.15.188:3300/0 conn(0x7f5bd40b0f10 0x7f5bd40b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/662329039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40cecc0 0x7f5bd40b22e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 -- 172.21.15.3:0/662329039 >> 172.21.15.3:0/662329039 conn(0x7f5bd406d4e0 msgr2=0x7f5bd406ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:08.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 -- 172.21.15.3:0/662329039 shutdown_connections 2023-11-04T05:29:08.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.845+0000 7f5bd9c04700 1 -- 172.21.15.3:0/662329039 wait complete. 2023-11-04T05:29:08.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.846+0000 7f5bd9c04700 1 Processor -- start 2023-11-04T05:29:08.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.846+0000 7f5bd9c04700 1 -- start start 2023-11-04T05:29:08.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd9c04700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40b0f10 0x7f5bd41f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd9c04700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 0x7f5bd40b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd9c04700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bd40b3b60 0x7f5bd40b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd9c04700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b4c80 con 0x7f5bd40cecc0 2023-11-04T05:29:08.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd9c04700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b4070 con 0x7f5bd40b3b60 2023-11-04T05:29:08.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd9c04700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b4350 con 0x7f5bd40b0f10 2023-11-04T05:29:08.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 0x7f5bd40b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 0x7f5bd40b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52800/0 (socket says 172.21.15.3:52800) 2023-11-04T05:29:08.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/2913423445 learned_addr learned my addr 172.21.15.3:0/2913423445 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:08.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.847+0000 7f5bd37fe700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40b0f10 0x7f5bd41f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd3fff700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bd40b3b60 0x7f5bd40b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40b0f10 msgr2=0x7f5bd41f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd2ffd700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40b0f10 0x7f5bd41f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bd40b3b60 msgr2=0x7f5bd40b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd2ffd700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bd40b3b60 0x7f5bd40b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd2ffd700 1 -- 172.21.15.3:0/2913423445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009550 con 0x7f5bd40cecc0 2023-11-04T05:29:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd3fff700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bd40b3b60 0x7f5bd40b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd2ffd700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 0x7f5bd40b3620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009c40 tx=0x7f5bc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40036a0 con 0x7f5bd40cecc0 2023-11-04T05:29:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc401e070 con 0x7f5bd40cecc0 2023-11-04T05:29:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd40b4630 con 0x7f5bd40cecc0 2023-11-04T05:29:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003a40 con 0x7f5bd40cecc0 2023-11-04T05:29:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.848+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd41f6570 con 0x7f5bd40cecc0 2023-11-04T05:29:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.849+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4074df0 con 0x7f5bd40cecc0 2023-11-04T05:29:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.849+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bc4003ba0 con 0x7f5bd40cecc0 2023-11-04T05:29:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.850+0000 7f5bcbfff700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5bb406af00 0x7f5bb406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.850+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5bc408a920 con 0x7f5bd40cecc0 2023-11-04T05:29:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.850+0000 7f5bd37fe700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5bb406af00 0x7f5bb406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.850+0000 7f5bd37fe700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5bb406af00 0x7f5bb406d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009b70 tx=0x7f5bbc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:08.854+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5bc40556a0 con 0x7f5bd40cecc0 2023-11-04T05:29:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.057+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5bd415d610 con 0x7f5bd40cecc0 2023-11-04T05:29:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.057+0000 7f5bcbfff700 1 -- 172.21.15.3:0/2913423445 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5bc4020020 con 0x7f5bd40cecc0 2023-11-04T05:29:09.060 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5bb406af00 msgr2=0x7f5bb406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5bb406af00 0x7f5bb406d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009b70 tx=0x7f5bbc008040 comp rx=0 tx=0).stop 2023-11-04T05:29:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 msgr2=0x7f5bd40b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 0x7f5bd40b3620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009c40 tx=0x7f5bc4004750 comp rx=0 tx=0).stop 2023-11-04T05:29:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 shutdown_connections 2023-11-04T05:29:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5bb406af00 0x7f5bb406d3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bd40b3b60 0x7f5bd40b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bd40b0f10 0x7f5bd41f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 --2- 172.21.15.3:0/2913423445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd40cecc0 0x7f5bd40b3620 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 >> 172.21.15.3:0/2913423445 conn(0x7f5bd406d4e0 msgr2=0x7f5bd40af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 shutdown_connections 2023-11-04T05:29:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.060+0000 7f5bd9c04700 1 -- 172.21.15.3:0/2913423445 wait complete. 2023-11-04T05:29:09.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.201+0000 7f080a7b8700 1 Processor -- start 2023-11-04T05:29:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.201+0000 7f080a7b8700 1 -- start start 2023-11-04T05:29:09.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f080a7b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 0x7f080415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f080a7b8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08040ccc70 con 0x7f080415b260 2023-11-04T05:29:09.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f080a7b8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080415baa0 con 0x7f08040ceaa0 2023-11-04T05:29:09.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f080a7b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08040ccec0 con 0x7f08040cee80 2023-11-04T05:29:09.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f0808d55700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f080415b260 0x7f08040caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58158/0 (socket says 172.21.15.3:58158) 2023-11-04T05:29:09.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f0808d55700 1 -- 172.21.15.3:0/476606069 learned_addr learned my addr 172.21.15.3:0/476606069 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:09.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f08037fe700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 0x7f080415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.202+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871297000 0 0) 0x7f08040ccc70 con 0x7f080415b260 2023-11-04T05:29:09.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f00057c0 con 0x7f080415b260 2023-11-04T05:29:09.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602517084 0 0) 0x7f07f00057c0 con 0x7f080415b260 2023-11-04T05:29:09.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 msgr2=0x7f080415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:09.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 0x7f080415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 >> v1:172.21.15.188:6789/0 conn(0x7f08040ceaa0 legacy=0x7f08040cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040cd1a0 con 0x7f080415b260 2023-11-04T05:29:09.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f07fc0019f0 con 0x7f080415b260 2023-11-04T05:29:09.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 >> v1:172.21.15.3:6789/0 conn(0x7f080415b260 legacy=0x7f08040caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f0005c40 0x7f07f0006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 0x7f07f0006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f0006fa0 0x7f07f0007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040ccec0 con 0x7f07f0006fa0 2023-11-04T05:29:09.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f000dcd0 con 0x7f07f0005c40 2023-11-04T05:29:09.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f000de90 con 0x7f08040cee80 2023-11-04T05:29:09.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f0006fa0 0x7f07f0007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f08037fe700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 0x7f07f0006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0803fff700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f0005c40 0x7f07f0006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 -- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 msgr2=0x7f07f0006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:09.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040cee80 0x7f07f0006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 -- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f0005c40 msgr2=0x7f07f0006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:09.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f0005c40 0x7f07f0006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 -- 172.21.15.3:0/476606069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00057c0 con 0x7f07f0006fa0 2023-11-04T05:29:09.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.203+0000 7f0808d55700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f0006fa0 0x7f07f0007420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00a7a0 tx=0x7f07f0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fde48dc758b4e2c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc012600 con 0x7f07f0006fa0 2023-11-04T05:29:09.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc012760 con 0x7f07f0006fa0 2023-11-04T05:29:09.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f0802ffd700 1 -- 172.21.15.3:0/476606069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc012af0 con 0x7f07f0006fa0 2023-11-04T05:29:09.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f080a7b8700 1 -- 172.21.15.3:0/476606069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f0006fa0 msgr2=0x7f07f0007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f080a7b8700 1 --2- 172.21.15.3:0/476606069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07f0006fa0 0x7f07f0007420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00a7a0 tx=0x7f07f0007a90 comp rx=0 tx=0).stop 2023-11-04T05:29:09.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f0808d55700 1 -- 172.21.15.3:0/476606069 reap_dead start 2023-11-04T05:29:09.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f080a7b8700 1 -- 172.21.15.3:0/476606069 shutdown_connections 2023-11-04T05:29:09.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f080a7b8700 1 -- 172.21.15.3:0/476606069 >> 172.21.15.3:0/476606069 conn(0x7f0804072560 msgr2=0x7f0804099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:09.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f080a7b8700 1 -- 172.21.15.3:0/476606069 shutdown_connections 2023-11-04T05:29:09.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.204+0000 7f080a7b8700 1 -- 172.21.15.3:0/476606069 wait complete. 2023-11-04T05:29:09.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.205+0000 7f080a7b8700 1 Processor -- start 2023-11-04T05:29:09.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 -- start start 2023-11-04T05:29:09.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 0x7f08040c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040c98b0 0x7f0804159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08040c8450 0x7f0804159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040c7720 con 0x7f0804154c10 2023-11-04T05:29:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040c9df0 con 0x7f08040c8450 2023-11-04T05:29:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.206+0000 7f080a7b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08040c7970 con 0x7f08040c98b0 2023-11-04T05:29:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 0x7f08040c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 0x7f08040c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52818/0 (socket says 172.21.15.3:52818) 2023-11-04T05:29:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 -- 172.21.15.3:0/4279448303 learned_addr learned my addr 172.21.15.3:0/4279448303 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f08037fe700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040c98b0 0x7f0804159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0808d55700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08040c8450 0x7f0804159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 -- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040c98b0 msgr2=0x7f0804159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040c98b0 0x7f0804159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 -- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08040c8450 msgr2=0x7f0804159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08040c8450 0x7f0804159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 -- 172.21.15.3:0/4279448303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040c7c50 con 0x7f0804154c10 2023-11-04T05:29:09.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0803fff700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 0x7f08040c9370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07f400f860 tx=0x7f07f400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f40052e0 con 0x7f0804154c10 2023-11-04T05:29:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f4005460 con 0x7f0804154c10 2023-11-04T05:29:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f40038c0 con 0x7f0804154c10 2023-11-04T05:29:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08040c7f30 con 0x7f0804154c10 2023-11-04T05:29:09.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.207+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0804154ff0 con 0x7f0804154c10 2023-11-04T05:29:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.208+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08040c8170 con 0x7f0804154c10 2023-11-04T05:29:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.209+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07f4003a20 con 0x7f0804154c10 2023-11-04T05:29:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.209+0000 7f0800ff9700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f07e406af00 0x7f07e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.209+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f07f4089750 con 0x7f0804154c10 2023-11-04T05:29:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.209+0000 7f08037fe700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f07e406af00 0x7f07e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.210+0000 7f08037fe700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f07e406af00 0x7f07e406d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08040b6480 tx=0x7f07ec009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.213+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f08040c8170 con 0x7f0804154c10 2023-11-04T05:29:09.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.412+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08040ccc00 con 0x7f0804154c10 2023-11-04T05:29:09.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.413+0000 7f0800ff9700 1 -- 172.21.15.3:0/4279448303 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07f40544d0 con 0x7f0804154c10 2023-11-04T05:29:09.414 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:09.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f07e406af00 msgr2=0x7f07e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f07e406af00 0x7f07e406d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08040b6480 tx=0x7f07ec009210 comp rx=0 tx=0).stop 2023-11-04T05:29:09.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 msgr2=0x7f08040c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 0x7f08040c9370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07f400f860 tx=0x7f07f400fb70 comp rx=0 tx=0).stop 2023-11-04T05:29:09.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 shutdown_connections 2023-11-04T05:29:09.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f07e406af00 0x7f07e406d3c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08040c8450 0x7f0804159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804154c10 0x7f08040c9370 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 --2- 172.21.15.3:0/4279448303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08040c98b0 0x7f0804159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 >> 172.21.15.3:0/4279448303 conn(0x7f0804072560 msgr2=0x7f080406ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:09.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 shutdown_connections 2023-11-04T05:29:09.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.415+0000 7f080a7b8700 1 -- 172.21.15.3:0/4279448303 wait complete. 2023-11-04T05:29:09.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.556+0000 7f7ce4766700 1 Processor -- start 2023-11-04T05:29:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.557+0000 7f7ce4766700 1 -- start start 2023-11-04T05:29:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.557+0000 7f7ce4766700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7cdc15dd80 0x7f7cdc06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce4766700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc06d920 con 0x7f7cdc0b3820 2023-11-04T05:29:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce4766700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc162250 con 0x7f7cdc1616e0 2023-11-04T05:29:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce4766700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc06bee0 con 0x7f7cdc15dd80 2023-11-04T05:29:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2d03700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7cdc0b3820 0x7f7cdc0cbce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58178/0 (socket says 172.21.15.3:58178) 2023-11-04T05:29:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2d03700 1 -- 172.21.15.3:0/3293505614 learned_addr learned my addr 172.21.15.3:0/3293505614 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2502700 1 --2- 172.21.15.3:0/3293505614 >> v2:172.21.15.188:3300/0 conn(0x7f7cdc15dd80 0x7f7cdc06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce1500700 1 -- 172.21.15.3:0/3293505614 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107656507 0 0) 0x7f7cdc06d920 con 0x7f7cdc0b3820 2023-11-04T05:29:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce1500700 1 -- 172.21.15.3:0/3293505614 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc80035b0 con 0x7f7cdc0b3820 2023-11-04T05:29:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce1500700 1 -- 172.21.15.3:0/3293505614 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680603384 0 0) 0x7f7cc80035b0 con 0x7f7cdc0b3820 2023-11-04T05:29:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2502700 1 -- 172.21.15.3:0/3293505614 >> v1:172.21.15.188:6789/0 conn(0x7f7cdc1616e0 legacy=0x7f7cdc0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2502700 1 -- 172.21.15.3:0/3293505614 >> v1:172.21.15.3:6789/0 conn(0x7f7cdc0b3820 legacy=0x7f7cdc0cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2502700 1 -- 172.21.15.3:0/3293505614 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc06c1c0 con 0x7f7cdc15dd80 2023-11-04T05:29:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.558+0000 7f7ce2502700 1 --2- 172.21.15.3:0/3293505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc15dd80 0x7f7cdc06c9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009840 tx=0x7f7cd0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e2f06a1ea31923 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce1500700 1 -- 172.21.15.3:0/3293505614 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40019f0 con 0x7f7cdc15dd80 2023-11-04T05:29:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce1500700 1 -- 172.21.15.3:0/3293505614 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0004030 con 0x7f7cdc15dd80 2023-11-04T05:29:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce1500700 1 -- 172.21.15.3:0/3293505614 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00043c0 con 0x7f7cdc15dd80 2023-11-04T05:29:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 -- 172.21.15.3:0/3293505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc15dd80 msgr2=0x7f7cdc06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 --2- 172.21.15.3:0/3293505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc15dd80 0x7f7cdc06c9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009840 tx=0x7f7cd0009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 -- 172.21.15.3:0/3293505614 shutdown_connections 2023-11-04T05:29:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 --2- 172.21.15.3:0/3293505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc15dd80 0x7f7cdc06c9c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 -- 172.21.15.3:0/3293505614 >> 172.21.15.3:0/3293505614 conn(0x7f7cdc0ce110 msgr2=0x7f7cdc0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 -- 172.21.15.3:0/3293505614 shutdown_connections 2023-11-04T05:29:09.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.559+0000 7f7ce4766700 1 -- 172.21.15.3:0/3293505614 wait complete. 2023-11-04T05:29:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.560+0000 7f7ce4766700 1 Processor -- start 2023-11-04T05:29:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 -- start start 2023-11-04T05:29:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0b3820 0x7f7cdc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 0x7f7cdc0adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc0c9cb0 0x7f7cdc15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0b0900 con 0x7f7cdc1616e0 2023-11-04T05:29:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0b43e0 con 0x7f7cdc0b3820 2023-11-04T05:29:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce4766700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0b0620 con 0x7f7cdc0c9cb0 2023-11-04T05:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce1d01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 0x7f7cdc0adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce1d01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 0x7f7cdc0adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52834/0 (socket says 172.21.15.3:52834) 2023-11-04T05:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce1d01700 1 -- 172.21.15.3:0/1380573220 learned_addr learned my addr 172.21.15.3:0/1380573220 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.561+0000 7f7ce2502700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0b3820 0x7f7cdc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce2d03700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc0c9cb0 0x7f7cdc15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce1d01700 1 -- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc0c9cb0 msgr2=0x7f7cdc15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce1d01700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc0c9cb0 0x7f7cdc15af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce1d01700 1 -- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0b3820 msgr2=0x7f7cdc0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce1d01700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0b3820 0x7f7cdc0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce1d01700 1 -- 172.21.15.3:0/1380573220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00094a0 con 0x7f7cdc1616e0 2023-11-04T05:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce1d01700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 0x7f7cdc0adad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800c450 tx=0x7f7cd800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80052d0 con 0x7f7cdc1616e0 2023-11-04T05:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8005430 con 0x7f7cdc1616e0 2023-11-04T05:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc0b4680 con 0x7f7cdc1616e0 2023-11-04T05:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80038c0 con 0x7f7cdc1616e0 2023-11-04T05:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.562+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc0b0240 con 0x7f7cdc1616e0 2023-11-04T05:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.563+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc06c9c0 con 0x7f7cdc1616e0 2023-11-04T05:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.564+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7cd8003a20 con 0x7f7cdc1616e0 2023-11-04T05:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.564+0000 7f7cceffd700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7cc006af00 0x7f7cc006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.564+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7cd8088ba0 con 0x7f7cdc1616e0 2023-11-04T05:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.564+0000 7f7ce2502700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7cc006af00 0x7f7cc006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.565+0000 7f7ce2502700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7cc006af00 0x7f7cc006d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00090b0 tx=0x7f7cd0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.568+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7cd8053920 con 0x7f7cdc1616e0 2023-11-04T05:29:09.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.769+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7cdc15c450 con 0x7f7cdc1616e0 2023-11-04T05:29:09.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.769+0000 7f7cceffd700 1 -- 172.21.15.3:0/1380573220 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7cd80170a0 con 0x7f7cdc1616e0 2023-11-04T05:29:09.771 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7cc006af00 msgr2=0x7f7cc006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7cc006af00 0x7f7cc006d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00090b0 tx=0x7f7cd0008030 comp rx=0 tx=0).stop 2023-11-04T05:29:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 msgr2=0x7f7cdc0adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 0x7f7cdc0adad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800c450 tx=0x7f7cd800c760 comp rx=0 tx=0).stop 2023-11-04T05:29:09.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 shutdown_connections 2023-11-04T05:29:09.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7cc006af00 0x7f7cc006d3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cdc0b3820 0x7f7cdc0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cdc0c9cb0 0x7f7cdc15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 --2- 172.21.15.3:0/1380573220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cdc1616e0 0x7f7cdc0adad0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 >> 172.21.15.3:0/1380573220 conn(0x7f7cdc0ce110 msgr2=0x7f7cdc0ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:09.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 shutdown_connections 2023-11-04T05:29:09.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.771+0000 7f7ce4766700 1 -- 172.21.15.3:0/1380573220 wait complete. 2023-11-04T05:29:09.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.910+0000 7fbdcedd2700 1 Processor -- start 2023-11-04T05:29:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.910+0000 7fbdcedd2700 1 -- start start 2023-11-04T05:29:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.910+0000 7fbdcedd2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbdc806ab20 0x7fbdc80728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.910+0000 7fbdcedd2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc8072df0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.910+0000 7fbdcedd2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc80cda00 con 0x7fbdc806af00 2023-11-04T05:29:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.910+0000 7fbdcedd2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80cf050 con 0x7fbdc806ab20 2023-11-04T05:29:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdc7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbdc80cc7a0 0x7fbdc80cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58190/0 (socket says 172.21.15.3:58190) 2023-11-04T05:29:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdc7fff700 1 -- 172.21.15.3:0/1864191931 learned_addr learned my addr 172.21.15.3:0/1864191931 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdccb6e700 1 --2- 172.21.15.3:0/1864191931 >> v2:172.21.15.188:3300/0 conn(0x7fbdc806ab20 0x7fbdc80728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdc77fe700 1 -- 172.21.15.3:0/1864191931 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222253343 0 0) 0x7fbdc8072df0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdc77fe700 1 -- 172.21.15.3:0/1864191931 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb80035b0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdccb6e700 1 -- 172.21.15.3:0/1864191931 >> v1:172.21.15.188:6789/0 conn(0x7fbdc806af00 legacy=0x7fbdc80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdccb6e700 1 -- 172.21.15.3:0/1864191931 >> v1:172.21.15.3:6789/0 conn(0x7fbdc80cc7a0 legacy=0x7fbdc80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdccb6e700 1 -- 172.21.15.3:0/1864191931 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80cdce0 con 0x7fbdc806ab20 2023-11-04T05:29:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.911+0000 7fbdccb6e700 1 --2- 172.21.15.3:0/1864191931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806ab20 0x7fbdc80728b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009840 tx=0x7fbdbc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada66fef5d66a90b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdc77fe700 1 -- 172.21.15.3:0/1864191931 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc011830 con 0x7fbdc806ab20 2023-11-04T05:29:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdc77fe700 1 -- 172.21.15.3:0/1864191931 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc011990 con 0x7fbdc806ab20 2023-11-04T05:29:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdc77fe700 1 -- 172.21.15.3:0/1864191931 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc011d00 con 0x7fbdc806ab20 2023-11-04T05:29:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 -- 172.21.15.3:0/1864191931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806ab20 msgr2=0x7fbdc80728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/1864191931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806ab20 0x7fbdc80728b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009840 tx=0x7fbdbc009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 -- 172.21.15.3:0/1864191931 shutdown_connections 2023-11-04T05:29:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/1864191931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806ab20 0x7fbdc80728b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 -- 172.21.15.3:0/1864191931 >> 172.21.15.3:0/1864191931 conn(0x7fbdc806df90 msgr2=0x7fbdc806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 -- 172.21.15.3:0/1864191931 shutdown_connections 2023-11-04T05:29:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.912+0000 7fbdcedd2700 1 -- 172.21.15.3:0/1864191931 wait complete. 2023-11-04T05:29:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.914+0000 7fbdcedd2700 1 Processor -- start 2023-11-04T05:29:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.914+0000 7fbdcedd2700 1 -- start start 2023-11-04T05:29:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcedd2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc806ab20 0x7fbdc81f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcedd2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806af00 0x7fbdc80c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcedd2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 0x7fbdc80c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcedd2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c5340 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcedd2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c5620 con 0x7fbdc806af00 2023-11-04T05:29:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcedd2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c5900 con 0x7fbdc806ab20 2023-11-04T05:29:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 0x7fbdc80c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 0x7fbdc80c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52850/0 (socket says 172.21.15.3:52850) 2023-11-04T05:29:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 -- 172.21.15.3:0/2888317427 learned_addr learned my addr 172.21.15.3:0/2888317427 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdc7fff700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806af00 0x7fbdc80c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdccb6e700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc806ab20 0x7fbdc81f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 -- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc806ab20 msgr2=0x7fbdc81f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc806ab20 0x7fbdc81f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 -- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806af00 msgr2=0x7fbdc80c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806af00 0x7fbdc80c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdcd36f700 1 -- 172.21.15.3:0/2888317427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0094a0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdccb6e700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc806ab20 0x7fbdc81f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.915+0000 7fbdc7fff700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806af00 0x7fbdc80c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.916+0000 7fbdcd36f700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 0x7fbdc80c6920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000cc50 tx=0x7fbdc000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.916+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000ad80 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.916+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc000ce50 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.916+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc80c5be0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.916+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003c40 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.916+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc8154ee0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.917+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc80c6100 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.917+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbdc00050d0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.917+0000 7fbdc57fa700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbdac06af00 0x7fbdac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:09.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.918+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbdc00899e0 con 0x7fbdc80cc7a0 2023-11-04T05:29:09.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.918+0000 7fbdccb6e700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbdac06af00 0x7fbdac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:09.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.918+0000 7fbdccb6e700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbdac06af00 0x7fbdac06d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0090b0 tx=0x7fbdbc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:09.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:09.922+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fbdc80c6100 con 0x7fbdc80cc7a0 2023-11-04T05:29:10.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.130+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdc80c6100 con 0x7fbdc80cc7a0 2023-11-04T05:29:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.131+0000 7fbdc57fa700 1 -- 172.21.15.3:0/2888317427 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7fbdc80c6100 con 0x7fbdc80cc7a0 2023-11-04T05:29:10.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:10.133 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbdac06af00 msgr2=0x7fbdac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbdac06af00 0x7fbdac06d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0090b0 tx=0x7fbdbc008030 comp rx=0 tx=0).stop 2023-11-04T05:29:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 msgr2=0x7fbdc80c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 0x7fbdc80c6920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000cc50 tx=0x7fbdc000a340 comp rx=0 tx=0).stop 2023-11-04T05:29:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 shutdown_connections 2023-11-04T05:29:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbdac06af00 0x7fbdac06d3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdc80cc7a0 0x7fbdc80c6920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc806ab20 0x7fbdc81f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 --2- 172.21.15.3:0/2888317427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc806af00 0x7fbdc80c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.133+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 >> 172.21.15.3:0/2888317427 conn(0x7fbdc806df90 msgr2=0x7fbdc806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:10.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.134+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 shutdown_connections 2023-11-04T05:29:10.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.134+0000 7fbdcedd2700 1 -- 172.21.15.3:0/2888317427 wait complete. 2023-11-04T05:29:10.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:10.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:10.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.275+0000 7f5a6d521700 1 Processor -- start 2023-11-04T05:29:10.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.275+0000 7f5a6d521700 1 -- start start 2023-11-04T05:29:10.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a6d521700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a680cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a6d521700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a6815b430 0x7f5a680caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a6d521700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680ccc70 con 0x7f5a680cee80 2023-11-04T05:29:10.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a6d521700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6815b8a0 con 0x7f5a6815b430 2023-11-04T05:29:10.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a6d521700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cdee0 con 0x7f5a680ceaa0 2023-11-04T05:29:10.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a667fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5a680cee80 0x7f5a6815ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58204/0 (socket says 172.21.15.3:58204) 2023-11-04T05:29:10.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a667fc700 1 -- 172.21.15.3:0/497887181 learned_addr learned my addr 172.21.15.3:0/497887181 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:10.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a677fe700 1 --2- 172.21.15.3:0/497887181 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815b430 0x7f5a680caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a680cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411207390 0 0) 0x7f5a680ccc70 con 0x7f5a680cee80 2023-11-04T05:29:10.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.276+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a500057c0 con 0x7f5a680cee80 2023-11-04T05:29:10.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3327588008 0 0) 0x7f5a500057c0 con 0x7f5a680cee80 2023-11-04T05:29:10.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 msgr2=0x7f5a680cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a680cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815b430 msgr2=0x7f5a680caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 --2- 172.21.15.3:0/497887181 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815b430 0x7f5a680caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6815baa0 con 0x7f5a680cee80 2023-11-04T05:29:10.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f5a58001670 con 0x7f5a680cee80 2023-11-04T05:29:10.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 >> v1:172.21.15.3:6789/0 conn(0x7f5a680cee80 legacy=0x7f5a6815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a50005c10 0x7f5a50006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50006a60 0x7f5a50006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a50007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6815b8a0 con 0x7f5a50005c10 2023-11-04T05:29:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cdee0 con 0x7f5a50006a60 2023-11-04T05:29:10.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000dc20 con 0x7f5a680ceaa0 2023-11-04T05:29:10.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a50005c10 0x7f5a50006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a667fc700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50006a60 0x7f5a50006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a677fe700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a50007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 -- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 msgr2=0x7f5a50007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a50007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 -- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50006a60 msgr2=0x7f5a50006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50006a60 0x7f5a50006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 -- 172.21.15.3:0/497887181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500057c0 con 0x7f5a50005c10 2023-11-04T05:29:10.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a677fe700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ceaa0 0x7f5a50007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:10.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a667fc700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a50006a60 0x7f5a50006ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:10.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a50005c10 0x7f5a50006030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009b90 tx=0x7f5a50007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fd4cebcd98d0bb server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00e6c0 con 0x7f5a50005c10 2023-11-04T05:29:10.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.277+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00e820 con 0x7f5a50005c10 2023-11-04T05:29:10.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a65ffb700 1 -- 172.21.15.3:0/497887181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00eb90 con 0x7f5a50005c10 2023-11-04T05:29:10.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a6d521700 1 -- 172.21.15.3:0/497887181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a50005c10 msgr2=0x7f5a50006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a6d521700 1 --2- 172.21.15.3:0/497887181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a50005c10 0x7f5a50006030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009b90 tx=0x7f5a50007a90 comp rx=0 tx=0).stop 2023-11-04T05:29:10.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a677fe700 1 -- 172.21.15.3:0/497887181 reap_dead start 2023-11-04T05:29:10.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a6d521700 1 -- 172.21.15.3:0/497887181 shutdown_connections 2023-11-04T05:29:10.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a6d521700 1 -- 172.21.15.3:0/497887181 >> 172.21.15.3:0/497887181 conn(0x7f5a68072560 msgr2=0x7f5a68099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:10.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a6d521700 1 -- 172.21.15.3:0/497887181 shutdown_connections 2023-11-04T05:29:10.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.278+0000 7f5a6d521700 1 -- 172.21.15.3:0/497887181 wait complete. 2023-11-04T05:29:10.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 Processor -- start 2023-11-04T05:29:10.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 -- start start 2023-11-04T05:29:10.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ae3e0 0x7f5a681f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680c2e90 0x7f5a680b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 0x7f5a6815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b1ad0 con 0x7f5a680b1640 2023-11-04T05:29:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68153990 con 0x7f5a680c2e90 2023-11-04T05:29:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a6d521700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681f3dd0 con 0x7f5a680ae3e0 2023-11-04T05:29:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.280+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 0x7f5a6815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 0x7f5a6815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52870/0 (socket says 172.21.15.3:52870) 2023-11-04T05:29:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 -- 172.21.15.3:0/2706320333 learned_addr learned my addr 172.21.15.3:0/2706320333 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a667fc700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680c2e90 0x7f5a680b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ae3e0 0x7f5a681f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 -- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ae3e0 msgr2=0x7f5a681f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ae3e0 0x7f5a681f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 -- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680c2e90 msgr2=0x7f5a680b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680c2e90 0x7f5a680b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 -- 172.21.15.3:0/2706320333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a681f40b0 con 0x7f5a680b1640 2023-11-04T05:29:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a667fc700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680c2e90 0x7f5a680b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ae3e0 0x7f5a681f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a677fe700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 0x7f5a6815ce60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a6815b810 tx=0x7f5a600104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6001b070 con 0x7f5a680b1640 2023-11-04T05:29:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000f040 con 0x7f5a680b1640 2023-11-04T05:29:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a681f4390 con 0x7f5a680b1640 2023-11-04T05:29:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000e050 con 0x7f5a680b1640 2023-11-04T05:29:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.281+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a680ae7c0 con 0x7f5a680b1640 2023-11-04T05:29:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.282+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6809e410 con 0x7f5a680b1640 2023-11-04T05:29:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.283+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a60015440 con 0x7f5a680b1640 2023-11-04T05:29:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.283+0000 7f5a4f7fe700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5a5406af00 0x7f5a5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.283+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a60089930 con 0x7f5a680b1640 2023-11-04T05:29:10.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.283+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5a5406af00 0x7f5a5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.284+0000 7f5a66ffd700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5a5406af00 0x7f5a5406d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0090b0 tx=0x7f5a5c01e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.288+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5a60054620 con 0x7f5a680b1640 2023-11-04T05:29:10.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.487+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a681571d0 con 0x7f5a680b1640 2023-11-04T05:29:10.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.488+0000 7f5a4f7fe700 1 -- 172.21.15.3:0/2706320333 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a6005e070 con 0x7f5a680b1640 2023-11-04T05:29:10.489 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:10.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5a5406af00 msgr2=0x7f5a5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5a5406af00 0x7f5a5406d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0090b0 tx=0x7f5a5c01e040 comp rx=0 tx=0).stop 2023-11-04T05:29:10.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 msgr2=0x7f5a6815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 0x7f5a6815ce60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a6815b810 tx=0x7f5a600104a0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 shutdown_connections 2023-11-04T05:29:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5a5406af00 0x7f5a5406d3c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a680b1640 0x7f5a6815ce60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680ae3e0 0x7f5a681f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 --2- 172.21.15.3:0/2706320333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680c2e90 0x7f5a680b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 >> 172.21.15.3:0/2706320333 conn(0x7f5a68072560 msgr2=0x7f5a680736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 shutdown_connections 2023-11-04T05:29:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.490+0000 7f5a6d521700 1 -- 172.21.15.3:0/2706320333 wait complete. 2023-11-04T05:29:10.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:10.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 Processor -- start 2023-11-04T05:29:10.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 -- start start 2023-11-04T05:29:10.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 0x7ffad4066cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffad40cdce0 0x7ffad406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad4067200 con 0x7ffad406f3f0 2023-11-04T05:29:10.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad415ceb0 con 0x7ffad40cdce0 2023-11-04T05:29:10.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.631+0000 7ffadaa41700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4156430 con 0x7ffad40668e0 2023-11-04T05:29:10.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad8fde700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffad406f3f0 0x7ffad415f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58224/0 (socket says 172.21.15.3:58224) 2023-11-04T05:29:10.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad8fde700 1 -- 172.21.15.3:0/3299471761 learned_addr learned my addr 172.21.15.3:0/3299471761 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:10.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad3fff700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 0x7ffad4066cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad37fe700 1 --2- 172.21.15.3:0/3299471761 >> v2:172.21.15.188:3300/0 conn(0x7ffad40cdce0 0x7ffad406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109756671 0 0) 0x7ffad4067200 con 0x7ffad406f3f0 2023-11-04T05:29:10.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffabc0035b0 con 0x7ffad406f3f0 2023-11-04T05:29:10.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364941163 0 0) 0x7ffabc0035b0 con 0x7ffad406f3f0 2023-11-04T05:29:10.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 msgr2=0x7ffad4066cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 0x7ffad4066cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 >> v2:172.21.15.188:3300/0 conn(0x7ffad40cdce0 msgr2=0x7ffad406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 --2- 172.21.15.3:0/3299471761 >> v2:172.21.15.188:3300/0 conn(0x7ffad40cdce0 0x7ffad406eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad415c330 con 0x7ffad406f3f0 2023-11-04T05:29:10.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7ffacc0019f0 con 0x7ffad406f3f0 2023-11-04T05:29:10.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 >> v1:172.21.15.3:6789/0 conn(0x7ffad406f3f0 legacy=0x7ffad415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffabc003a00 0x7ffabc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 0x7ffabc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc009a00 0x7ffabc004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad415ceb0 con 0x7ffabc003a00 2023-11-04T05:29:10.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4156430 con 0x7ffabc009a00 2023-11-04T05:29:10.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc005530 con 0x7ffad40668e0 2023-11-04T05:29:10.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.632+0000 7ffad3fff700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffabc003a00 0x7ffabc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad8fde700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc009a00 0x7ffabc004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad37fe700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 0x7ffabc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad3fff700 1 -- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 msgr2=0x7ffabc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad3fff700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40668e0 0x7ffabc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad3fff700 1 -- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc009a00 msgr2=0x7ffabc004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:10.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad3fff700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc009a00 0x7ffabc004d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad3fff700 1 -- 172.21.15.3:0/3299471761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0035b0 con 0x7ffabc003a00 2023-11-04T05:29:10.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad3fff700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffabc003a00 0x7ffabc003e20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009b60 tx=0x7ffabc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4971ef9e888eb140 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400e6c0 con 0x7ffabc003a00 2023-11-04T05:29:10.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400e820 con 0x7ffabc003a00 2023-11-04T05:29:10.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.633+0000 7ffad2ffd700 1 -- 172.21.15.3:0/3299471761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400eb90 con 0x7ffabc003a00 2023-11-04T05:29:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffadaa41700 1 -- 172.21.15.3:0/3299471761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffabc003a00 msgr2=0x7ffabc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffadaa41700 1 --2- 172.21.15.3:0/3299471761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffabc003a00 0x7ffabc003e20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009b60 tx=0x7ffabc005880 comp rx=0 tx=0).stop 2023-11-04T05:29:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffad8fde700 1 -- 172.21.15.3:0/3299471761 reap_dead start 2023-11-04T05:29:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffadaa41700 1 -- 172.21.15.3:0/3299471761 shutdown_connections 2023-11-04T05:29:10.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffadaa41700 1 -- 172.21.15.3:0/3299471761 >> 172.21.15.3:0/3299471761 conn(0x7ffad40cc860 msgr2=0x7ffad40ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:10.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffadaa41700 1 -- 172.21.15.3:0/3299471761 shutdown_connections 2023-11-04T05:29:10.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.634+0000 7ffadaa41700 1 -- 172.21.15.3:0/3299471761 wait complete. 2023-11-04T05:29:10.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.635+0000 7ffadaa41700 1 Processor -- start 2023-11-04T05:29:10.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.635+0000 7ffadaa41700 1 -- start start 2023-11-04T05:29:10.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffadaa41700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc003a00 0x7ffad41565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffadaa41700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40ae570 0x7ffad4142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffadaa41700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 0x7ffad4143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffadaa41700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad41439c0 con 0x7ffad40ae9f0 2023-11-04T05:29:10.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffadaa41700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4145500 con 0x7ffabc003a00 2023-11-04T05:29:10.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffadaa41700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4145990 con 0x7ffad40ae570 2023-11-04T05:29:10.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 0x7ffad4143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 0x7ffad4143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52888/0 (socket says 172.21.15.3:52888) 2023-11-04T05:29:10.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 -- 172.21.15.3:0/1668600601 learned_addr learned my addr 172.21.15.3:0/1668600601 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:10.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad37fe700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40ae570 0x7ffad4142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad3fff700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc003a00 0x7ffad41565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 -- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40ae570 msgr2=0x7ffad4142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40ae570 0x7ffad4142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 -- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc003a00 msgr2=0x7ffad41565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc003a00 0x7ffad41565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 -- 172.21.15.3:0/1668600601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4144610 con 0x7ffad40ae9f0 2023-11-04T05:29:10.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.636+0000 7ffad8fde700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 0x7ffad4143480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffacc001330 tx=0x7ffacc00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.637+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00ad50 con 0x7ffad40ae9f0 2023-11-04T05:29:10.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.637+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc0133f0 con 0x7ffad40ae9f0 2023-11-04T05:29:10.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.637+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad41448f0 con 0x7ffad40ae9f0 2023-11-04T05:29:10.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.637+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0137a0 con 0x7ffad40ae9f0 2023-11-04T05:29:10.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.637+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad4144eb0 con 0x7ffad40ae9f0 2023-11-04T05:29:10.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.638+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad4145640 con 0x7ffad40ae9f0 2023-11-04T05:29:10.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.638+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffacc013900 con 0x7ffad40ae9f0 2023-11-04T05:29:10.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.639+0000 7ffad0ff9700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffac006af00 0x7ffac006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.639+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffacc08a110 con 0x7ffad40ae9f0 2023-11-04T05:29:10.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.639+0000 7ffad3fff700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffac006af00 0x7ffac006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.639+0000 7ffad3fff700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffac006af00 0x7ffac006d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffac40090b0 tx=0x7ffac4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.643+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ffacc054e00 con 0x7ffad40ae9f0 2023-11-04T05:29:10.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.844+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffad406e2b0 con 0x7ffad40ae9f0 2023-11-04T05:29:10.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.844+0000 7ffad0ff9700 1 -- 172.21.15.3:0/1668600601 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffacc058420 con 0x7ffad40ae9f0 2023-11-04T05:29:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffac006af00 msgr2=0x7ffac006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffac006af00 0x7ffac006d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffac40090b0 tx=0x7ffac4008030 comp rx=0 tx=0).stop 2023-11-04T05:29:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 msgr2=0x7ffad4143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 0x7ffad4143480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffacc001330 tx=0x7ffacc00a640 comp rx=0 tx=0).stop 2023-11-04T05:29:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 shutdown_connections 2023-11-04T05:29:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ffac006af00 0x7ffac006d3c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc003a00 0x7ffad41565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffad40ae9f0 0x7ffad4143480 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 --2- 172.21.15.3:0/1668600601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad40ae570 0x7ffad4142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 >> 172.21.15.3:0/1668600601 conn(0x7ffad40cc860 msgr2=0x7ffad40ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 shutdown_connections 2023-11-04T05:29:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.846+0000 7ffadaa41700 1 -- 172.21.15.3:0/1668600601 wait complete. 2023-11-04T05:29:10.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.987+0000 7f1203558700 1 Processor -- start 2023-11-04T05:29:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.987+0000 7f1203558700 1 -- start start 2023-11-04T05:29:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f1203558700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11fc1611d0 0x7f11fc0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f1203558700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0caca0 con 0x7f11fc0cd4b0 2023-11-04T05:29:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f1203558700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0cfe40 con 0x7f11fc06b230 2023-11-04T05:29:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f1203558700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc0ce0a0 con 0x7f11fc1611d0 2023-11-04T05:29:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f1200af3700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f11fc0cd4b0 0x7f11fc06ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58240/0 (socket says 172.21.15.3:58240) 2023-11-04T05:29:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f1200af3700 1 -- 172.21.15.3:0/3756798839 learned_addr learned my addr 172.21.15.3:0/3756798839 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.988+0000 7f12012f4700 1 --2- 172.21.15.3:0/3756798839 >> v2:172.21.15.188:3300/0 conn(0x7f11fc1611d0 0x7f11fc0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f11f3fff700 1 -- 172.21.15.3:0/3756798839 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706056798 0 0) 0x7f11fc0caca0 con 0x7f11fc0cd4b0 2023-11-04T05:29:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f11f3fff700 1 -- 172.21.15.3:0/3756798839 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e80035b0 con 0x7f11fc0cd4b0 2023-11-04T05:29:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f12012f4700 1 -- 172.21.15.3:0/3756798839 >> v1:172.21.15.188:6789/0 conn(0x7f11fc06b230 legacy=0x7f11fc0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f12012f4700 1 -- 172.21.15.3:0/3756798839 >> v1:172.21.15.3:6789/0 conn(0x7f11fc0cd4b0 legacy=0x7f11fc06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f12012f4700 1 -- 172.21.15.3:0/3756798839 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0cf850 con 0x7f11fc1611d0 2023-11-04T05:29:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f12012f4700 1 --2- 172.21.15.3:0/3756798839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc1611d0 0x7f11fc0ccf70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009840 tx=0x7f11ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656598104c32a71 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f11f3fff700 1 -- 172.21.15.3:0/3756798839 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec011830 con 0x7f11fc1611d0 2023-11-04T05:29:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f11f3fff700 1 -- 172.21.15.3:0/3756798839 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec011990 con 0x7f11fc1611d0 2023-11-04T05:29:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.989+0000 7f11f3fff700 1 -- 172.21.15.3:0/3756798839 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec011d20 con 0x7f11fc1611d0 2023-11-04T05:29:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 -- 172.21.15.3:0/3756798839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc1611d0 msgr2=0x7f11fc0ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 --2- 172.21.15.3:0/3756798839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc1611d0 0x7f11fc0ccf70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009840 tx=0x7f11ec009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 -- 172.21.15.3:0/3756798839 shutdown_connections 2023-11-04T05:29:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 --2- 172.21.15.3:0/3756798839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc1611d0 0x7f11fc0ccf70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 -- 172.21.15.3:0/3756798839 >> 172.21.15.3:0/3756798839 conn(0x7f11fc072560 msgr2=0x7f11fc099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 -- 172.21.15.3:0/3756798839 shutdown_connections 2023-11-04T05:29:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.990+0000 7f1203558700 1 -- 172.21.15.3:0/3756798839 wait complete. 2023-11-04T05:29:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.991+0000 7f1203558700 1 Processor -- start 2023-11-04T05:29:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.991+0000 7f1203558700 1 -- start start 2023-11-04T05:29:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc06b230 0x7f11fc15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11fc0cd4b0 0x7f11fc1f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 0x7f11fc1f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15cc80 con 0x7f11fc1611d0 2023-11-04T05:29:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0ad450 con 0x7f11fc06b230 2023-11-04T05:29:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15fc10 con 0x7f11fc0cd4b0 2023-11-04T05:29:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 0x7f11fc1f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 0x7f11fc1f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52902/0 (socket says 172.21.15.3:52902) 2023-11-04T05:29:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 -- 172.21.15.3:0/1655003950 learned_addr learned my addr 172.21.15.3:0/1655003950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f12012f4700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc06b230 0x7f11fc15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1200af3700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11fc0cd4b0 0x7f11fc1f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 -- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11fc0cd4b0 msgr2=0x7f11fc1f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11fc0cd4b0 0x7f11fc1f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 -- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc06b230 msgr2=0x7f11fc15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc06b230 0x7f11fc15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 -- 172.21.15.3:0/1655003950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0094a0 con 0x7f11fc1611d0 2023-11-04T05:29:11.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1201af5700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 0x7f11fc1f6e40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11f400cbf0 tx=0x7f11f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4003c00 con 0x7f11fc1611d0 2023-11-04T05:29:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4003d60 con 0x7f11fc1611d0 2023-11-04T05:29:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc15fef0 con 0x7f11fc1611d0 2023-11-04T05:29:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.992+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400ac20 con 0x7f11fc1611d0 2023-11-04T05:29:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.993+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1604b0 con 0x7f11fc1611d0 2023-11-04T05:29:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.994+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc15daf0 con 0x7f11fc1611d0 2023-11-04T05:29:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.994+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11f40050d0 con 0x7f11fc1611d0 2023-11-04T05:29:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.994+0000 7f11f1ffb700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f11e006af00 0x7f11e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.994+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f11f40899e0 con 0x7f11fc1611d0 2023-11-04T05:29:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.995+0000 7f12012f4700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f11e006af00 0x7f11e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.995+0000 7f12012f4700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f11e006af00 0x7f11e006d3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0090b0 tx=0x7f11ec01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:10.999+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f11f4054760 con 0x7f11fc1611d0 2023-11-04T05:29:11.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.203+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11fc15d810 con 0x7f11fc1611d0 2023-11-04T05:29:11.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.203+0000 7f11f1ffb700 1 -- 172.21.15.3:0/1655003950 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11f4057d80 con 0x7f11fc1611d0 2023-11-04T05:29:11.205 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f11e006af00 msgr2=0x7f11e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f11e006af00 0x7f11e006d3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0090b0 tx=0x7f11ec01c040 comp rx=0 tx=0).stop 2023-11-04T05:29:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 msgr2=0x7f11fc1f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 0x7f11fc1f6e40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11f400cbf0 tx=0x7f11f400a060 comp rx=0 tx=0).stop 2023-11-04T05:29:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 shutdown_connections 2023-11-04T05:29:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f11e006af00 0x7f11e006d3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11fc06b230 0x7f11fc15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc1611d0 0x7f11fc1f6e40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 --2- 172.21.15.3:0/1655003950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11fc0cd4b0 0x7f11fc1f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 >> 172.21.15.3:0/1655003950 conn(0x7f11fc072560 msgr2=0x7f11fc0cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 shutdown_connections 2023-11-04T05:29:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.205+0000 7f1203558700 1 -- 172.21.15.3:0/1655003950 wait complete. 2023-11-04T05:29:11.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.345+0000 7f2df49f6700 1 Processor -- start 2023-11-04T05:29:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.346+0000 7f2df49f6700 1 -- start start 2023-11-04T05:29:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.346+0000 7f2df49f6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2df006ab20 0x7f2df006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2df49f6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2df00d02d0 0x7f2df00d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2df49f6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df00caca0 con 0x7f2df01611d0 2023-11-04T05:29:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2df49f6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df00ce0a0 con 0x7f2df00d02d0 2023-11-04T05:29:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2df49f6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df006b4e0 con 0x7f2df006ab20 2023-11-04T05:29:11.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2dee59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2df01611d0 0x7f2df00ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58254/0 (socket says 172.21.15.3:58254) 2023-11-04T05:29:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2deed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2df00d02d0 0x7f2df00d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2dee59c700 1 -- 172.21.15.3:0/3292886951 learned_addr learned my addr 172.21.15.3:0/3292886951 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2dedd9b700 1 --2- 172.21.15.3:0/3292886951 >> v2:172.21.15.188:3300/0 conn(0x7f2df006ab20 0x7f2df006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2ded59a700 1 -- 172.21.15.3:0/3292886951 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773255970 0 0) 0x7f2df00caca0 con 0x7f2df01611d0 2023-11-04T05:29:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2ded59a700 1 -- 172.21.15.3:0/3292886951 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd80035b0 con 0x7f2df01611d0 2023-11-04T05:29:11.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2deed9d700 1 -- 172.21.15.3:0/3292886951 >> v2:172.21.15.188:3300/0 conn(0x7f2df006ab20 msgr2=0x7f2df006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:11.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2deed9d700 1 --2- 172.21.15.3:0/3292886951 >> v2:172.21.15.188:3300/0 conn(0x7f2df006ab20 0x7f2df006afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2deed9d700 1 -- 172.21.15.3:0/3292886951 >> v1:172.21.15.3:6789/0 conn(0x7f2df01611d0 legacy=0x7f2df00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2deed9d700 1 -- 172.21.15.3:0/3292886951 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00cfe40 con 0x7f2df00d02d0 2023-11-04T05:29:11.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2deed9d700 1 --2- 172.21.15.3:0/3292886951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df00d02d0 0x7f2df00d0760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2df00c7680 tx=0x7f2de800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6cf1e0e145c687 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2ded59a700 1 -- 172.21.15.3:0/3292886951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800e070 con 0x7f2df00d02d0 2023-11-04T05:29:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2ded59a700 1 -- 172.21.15.3:0/3292886951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de800bb20 con 0x7f2df00d02d0 2023-11-04T05:29:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.347+0000 7f2ded59a700 1 -- 172.21.15.3:0/3292886951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800be90 con 0x7f2df00d02d0 2023-11-04T05:29:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 -- 172.21.15.3:0/3292886951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df00d02d0 msgr2=0x7f2df00d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 --2- 172.21.15.3:0/3292886951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df00d02d0 0x7f2df00d0760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2df00c7680 tx=0x7f2de800b330 comp rx=0 tx=0).stop 2023-11-04T05:29:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 -- 172.21.15.3:0/3292886951 shutdown_connections 2023-11-04T05:29:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 --2- 172.21.15.3:0/3292886951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df00d02d0 0x7f2df00d0760 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 --2- 172.21.15.3:0/3292886951 >> v2:172.21.15.188:3300/0 conn(0x7f2df006ab20 0x7f2df006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 -- 172.21.15.3:0/3292886951 >> 172.21.15.3:0/3292886951 conn(0x7f2df0072560 msgr2=0x7f2df0099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 -- 172.21.15.3:0/3292886951 shutdown_connections 2023-11-04T05:29:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.348+0000 7f2df49f6700 1 -- 172.21.15.3:0/3292886951 wait complete. 2023-11-04T05:29:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.349+0000 7f2df49f6700 1 Processor -- start 2023-11-04T05:29:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.349+0000 7f2df49f6700 1 -- start start 2023-11-04T05:29:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2df49f6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 0x7f2df015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2df49f6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df00d02d0 0x7f2df015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2df49f6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df01611d0 0x7f2df015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2df49f6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00c5f00 con 0x7f2df006ab20 2023-11-04T05:29:11.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2df49f6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df015f8c0 con 0x7f2df00d02d0 2023-11-04T05:29:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2df49f6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df015beb0 con 0x7f2df01611d0 2023-11-04T05:29:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 0x7f2df015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2dee59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 0x7f2df015e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52912/0 (socket says 172.21.15.3:52912) 2023-11-04T05:29:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2dee59c700 1 -- 172.21.15.3:0/2637315569 learned_addr learned my addr 172.21.15.3:0/2637315569 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2deed9d700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df01611d0 0x7f2df015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.350+0000 7f2dedd9b700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df00d02d0 0x7f2df015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dee59c700 1 -- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df01611d0 msgr2=0x7f2df015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dee59c700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df01611d0 0x7f2df015b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dee59c700 1 -- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df00d02d0 msgr2=0x7f2df015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dee59c700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df00d02d0 0x7f2df015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dee59c700 1 -- 172.21.15.3:0/2637315569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8009a70 con 0x7f2df006ab20 2023-11-04T05:29:11.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2deed9d700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df01611d0 0x7f2df015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:11.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dedd9b700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df00d02d0 0x7f2df015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2dee59c700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 0x7f2df015e4e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c420 tx=0x7f2de400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4005d70 con 0x7f2df006ab20 2023-11-04T05:29:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de40070d0 con 0x7f2df006ab20 2023-11-04T05:29:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df015cfe0 con 0x7f2df006ab20 2023-11-04T05:29:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4007440 con 0x7f2df006ab20 2023-11-04T05:29:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.351+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df00c4440 con 0x7f2df006ab20 2023-11-04T05:29:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.352+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df00c6040 con 0x7f2df006ab20 2023-11-04T05:29:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.352+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2de40075a0 con 0x7f2df006ab20 2023-11-04T05:29:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.353+0000 7f2ddeffd700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2dd006af00 0x7f2dd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.353+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2de4088cc0 con 0x7f2df006ab20 2023-11-04T05:29:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.353+0000 7f2dedd9b700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2dd006af00 0x7f2dd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.354+0000 7f2dedd9b700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2dd006af00 0x7f2dd006d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009b60 tx=0x7f2de0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.358+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2de4053a60 con 0x7f2df006ab20 2023-11-04T05:29:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.559+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2df006fef0 con 0x7f2df006ab20 2023-11-04T05:29:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.559+0000 7f2ddeffd700 1 -- 172.21.15.3:0/2637315569 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2de4017020 con 0x7f2df006ab20 2023-11-04T05:29:11.561 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:11.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2dd006af00 msgr2=0x7f2dd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2dd006af00 0x7f2dd006d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009b60 tx=0x7f2de0008040 comp rx=0 tx=0).stop 2023-11-04T05:29:11.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 msgr2=0x7f2df015e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 0x7f2df015e4e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c420 tx=0x7f2de400c7e0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 shutdown_connections 2023-11-04T05:29:11.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2dd006af00 0x7f2dd006d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df01611d0 0x7f2df015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2df006ab20 0x7f2df015e4e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 --2- 172.21.15.3:0/2637315569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df00d02d0 0x7f2df015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 >> 172.21.15.3:0/2637315569 conn(0x7f2df0072560 msgr2=0x7f2df006f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:11.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 shutdown_connections 2023-11-04T05:29:11.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.561+0000 7f2df49f6700 1 -- 172.21.15.3:0/2637315569 wait complete. 2023-11-04T05:29:11.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.698+0000 7ff9b5de7700 1 Processor -- start 2023-11-04T05:29:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.698+0000 7ff9b5de7700 1 -- start start 2023-11-04T05:29:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9b5de7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b00cc7a0 0x7ff9b00728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9b5de7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9b006af00 0x7ff9b00d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9b5de7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b0072df0 con 0x7ff9b006ab20 2023-11-04T05:29:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9b5de7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00cf050 con 0x7ff9b006af00 2023-11-04T05:29:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9b5de7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00cda00 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9b006af00 0x7ff9b00d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9b006af00 0x7ff9b00d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52924/0 (socket says 172.21.15.3:52924) 2023-11-04T05:29:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 -- 172.21.15.3:0/812166851 learned_addr learned my addr 172.21.15.3:0/812166851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9aeffd700 1 --2- 172.21.15.3:0/812166851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b00cc7a0 0x7ff9b00728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 -- 172.21.15.3:0/812166851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b00cc7a0 msgr2=0x7ff9b00728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 --2- 172.21.15.3:0/812166851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b00cc7a0 0x7ff9b00728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 -- 172.21.15.3:0/812166851 >> v1:172.21.15.188:6789/0 conn(0x7ff9b006ab20 legacy=0x7ff9b00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 -- 172.21.15.3:0/812166851 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00cdce0 con 0x7ff9b006af00 2023-11-04T05:29:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.699+0000 7ff9affff700 1 --2- 172.21.15.3:0/812166851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b006af00 0x7ff9b00d0440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009e80 tx=0x7ff9a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00dc20e8e4d281 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9ae7fc700 1 -- 172.21.15.3:0/812166851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800e070 con 0x7ff9b006af00 2023-11-04T05:29:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9ae7fc700 1 -- 172.21.15.3:0/812166851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8003a20 con 0x7ff9b006af00 2023-11-04T05:29:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9ae7fc700 1 -- 172.21.15.3:0/812166851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8003dd0 con 0x7ff9b006af00 2023-11-04T05:29:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 -- 172.21.15.3:0/812166851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b006af00 msgr2=0x7ff9b00d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/812166851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b006af00 0x7ff9b00d0440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009e80 tx=0x7ff9a800b330 comp rx=0 tx=0).stop 2023-11-04T05:29:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 -- 172.21.15.3:0/812166851 shutdown_connections 2023-11-04T05:29:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/812166851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b006af00 0x7ff9b00d0440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/812166851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b00cc7a0 0x7ff9b00728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 -- 172.21.15.3:0/812166851 >> 172.21.15.3:0/812166851 conn(0x7ff9b006df90 msgr2=0x7ff9b006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 -- 172.21.15.3:0/812166851 shutdown_connections 2023-11-04T05:29:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.700+0000 7ff9b5de7700 1 -- 172.21.15.3:0/812166851 wait complete. 2023-11-04T05:29:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.702+0000 7ff9b5de7700 1 Processor -- start 2023-11-04T05:29:11.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.702+0000 7ff9b5de7700 1 -- start start 2023-11-04T05:29:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9b5de7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b006ab20 0x7ff9b0158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9b5de7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9b006af00 0x7ff9b015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9b5de7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 0x7ff9b00c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9b5de7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00c7610 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9b5de7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0158e70 con 0x7ff9b006af00 2023-11-04T05:29:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9b5de7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00c7820 con 0x7ff9b006ab20 2023-11-04T05:29:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 0x7ff9b00c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 0x7ff9b00c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52930/0 (socket says 172.21.15.3:52930) 2023-11-04T05:29:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 -- 172.21.15.3:0/1175760262 learned_addr learned my addr 172.21.15.3:0/1175760262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9af7fe700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b006ab20 0x7ff9b0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9aeffd700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9b006af00 0x7ff9b015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 -- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b006ab20 msgr2=0x7ff9b0158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b006ab20 0x7ff9b0158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 -- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9b006af00 msgr2=0x7ff9b015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9b006af00 0x7ff9b015f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9affff700 1 -- 172.21.15.3:0/1175760262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8009b30 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.703+0000 7ff9af7fe700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b006ab20 0x7ff9b0158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.704+0000 7ff9affff700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 0x7ff9b00c4850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b730 tx=0x7ff9a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.704+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8004440 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.704+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a80045c0 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.704+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00c7b00 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.704+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8004970 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.704+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00c80c0 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.705+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b0074df0 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.705+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9a8012430 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.706+0000 7ff9a7fff700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff99006af00 0x7ff99006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.706+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff9a808ae30 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.706+0000 7ff9af7fe700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff99006af00 0x7ff99006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.706+0000 7ff9af7fe700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff99006af00 0x7ff99006d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff9b01e9d50 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:11.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.710+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff9a8055b30 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.912+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9b00cd020 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.913+0000 7ff9a7fff700 1 -- 172.21.15.3:0/1175760262 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7ff9a8059150 con 0x7ff9b00cc7a0 2023-11-04T05:29:11.914 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:11.915 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:11.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff99006af00 msgr2=0x7ff99006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff99006af00 0x7ff99006d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff9b01e9d50 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2023-11-04T05:29:11.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 msgr2=0x7ff9b00c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:11.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 0x7ff9b00c4850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b730 tx=0x7ff9a8004ad0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 shutdown_connections 2023-11-04T05:29:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff99006af00 0x7ff99006d3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9b00cc7a0 0x7ff9b00c4850 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b006ab20 0x7ff9b0158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 --2- 172.21.15.3:0/1175760262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9b006af00 0x7ff9b015f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:11.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 >> 172.21.15.3:0/1175760262 conn(0x7ff9b006df90 msgr2=0x7ff9b015bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:11.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 shutdown_connections 2023-11-04T05:29:11.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:11.915+0000 7ff9b5de7700 1 -- 172.21.15.3:0/1175760262 wait complete. 2023-11-04T05:29:11.929 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:11.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:12.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.058+0000 7f80937ed700 1 Processor -- start 2023-11-04T05:29:12.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.058+0000 7f80937ed700 1 -- start start 2023-11-04T05:29:12.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f80937ed700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f808c06ab20 0x7f808c06afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f80937ed700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0d02d0 0x7f808c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f80937ed700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c0caca0 con 0x7f808c1611d0 2023-11-04T05:29:12.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f80937ed700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808c0ce0a0 con 0x7f808c06ab20 2023-11-04T05:29:12.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f80937ed700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c06b4e0 con 0x7f808c0d02d0 2023-11-04T05:29:12.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f808c06ab20 0x7f808c06afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8091589700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f808c1611d0 0x7f808c0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58282/0 (socket says 172.21.15.3:58282) 2023-11-04T05:29:12.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f808c06ab20 0x7f808c06afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52936/0 (socket says 172.21.15.3:52936) 2023-11-04T05:29:12.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8091589700 1 -- 172.21.15.3:0/1433254415 learned_addr learned my addr 172.21.15.3:0/1433254415 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:12.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8091d8a700 1 --2- 172.21.15.3:0/1433254415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0d02d0 0x7f808c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 -- 172.21.15.3:0/1433254415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0d02d0 msgr2=0x7f808c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 --2- 172.21.15.3:0/1433254415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0d02d0 0x7f808c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 -- 172.21.15.3:0/1433254415 >> v1:172.21.15.3:6789/0 conn(0x7f808c1611d0 legacy=0x7f808c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 -- 172.21.15.3:0/1433254415 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0cfe40 con 0x7f808c06ab20 2023-11-04T05:29:12.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f807ffff700 1 -- 172.21.15.3:0/1433254415 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266387987 0 0) 0x7f808c0caca0 con 0x7f808c1611d0 2023-11-04T05:29:12.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f8090d88700 1 --2- 172.21.15.3:0/1433254415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c06ab20 0x7f808c06afa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8080009840 tx=0x7f8080009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658cd7b6ed1ab04f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f807ffff700 1 -- 172.21.15.3:0/1433254415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080011830 con 0x7f808c06ab20 2023-11-04T05:29:12.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f807ffff700 1 -- 172.21.15.3:0/1433254415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080011990 con 0x7f808c06ab20 2023-11-04T05:29:12.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.059+0000 7f807ffff700 1 -- 172.21.15.3:0/1433254415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080011d20 con 0x7f808c06ab20 2023-11-04T05:29:12.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 -- 172.21.15.3:0/1433254415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c06ab20 msgr2=0x7f808c06afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 --2- 172.21.15.3:0/1433254415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c06ab20 0x7f808c06afa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8080009840 tx=0x7f8080009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:12.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 -- 172.21.15.3:0/1433254415 shutdown_connections 2023-11-04T05:29:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 --2- 172.21.15.3:0/1433254415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0d02d0 0x7f808c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 --2- 172.21.15.3:0/1433254415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c06ab20 0x7f808c06afa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 -- 172.21.15.3:0/1433254415 >> 172.21.15.3:0/1433254415 conn(0x7f808c072560 msgr2=0x7f808c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 -- 172.21.15.3:0/1433254415 shutdown_connections 2023-11-04T05:29:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.060+0000 7f80937ed700 1 -- 172.21.15.3:0/1433254415 wait complete. 2023-11-04T05:29:12.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.061+0000 7f80937ed700 1 Processor -- start 2023-11-04T05:29:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.061+0000 7f80937ed700 1 -- start start 2023-11-04T05:29:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f80937ed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06ab20 0x7f808c1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f80937ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 0x7f808c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f80937ed700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c1611d0 0x7f808c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f80937ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c06fed0 con 0x7f808c0d02d0 2023-11-04T05:29:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f80937ed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1f34c0 con 0x7f808c06ab20 2023-11-04T05:29:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f80937ed700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c06f630 con 0x7f808c1611d0 2023-11-04T05:29:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 0x7f808c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 0x7f808c0b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52942/0 (socket says 172.21.15.3:52942) 2023-11-04T05:29:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 -- 172.21.15.3:0/1848279776 learned_addr learned my addr 172.21.15.3:0/1848279776 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8091589700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06ab20 0x7f808c1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8091d8a700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c1611d0 0x7f808c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 -- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c1611d0 msgr2=0x7f808c15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c1611d0 0x7f808c15b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 -- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06ab20 msgr2=0x7f808c1f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06ab20 0x7f808c1f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.062+0000 7f8090d88700 1 -- 172.21.15.3:0/1848279776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800094a0 con 0x7f808c0d02d0 2023-11-04T05:29:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f8091589700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06ab20 0x7f808c1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f8090d88700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 0x7f808c0b31b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f808c0d1750 tx=0x7f8080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800038a0 con 0x7f808c0d02d0 2023-11-04T05:29:12.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808001f070 con 0x7f808c0d02d0 2023-11-04T05:29:12.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080003c20 con 0x7f808c0d02d0 2023-11-04T05:29:12.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c06f910 con 0x7f808c0d02d0 2023-11-04T05:29:12.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.063+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c15b170 con 0x7f808c0d02d0 2023-11-04T05:29:12.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.064+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c15b390 con 0x7f808c0d02d0 2023-11-04T05:29:12.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.064+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f808001a430 con 0x7f808c0d02d0 2023-11-04T05:29:12.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.065+0000 7f807dffb700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f807006af00 0x7f807006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.065+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8080021070 con 0x7f808c0d02d0 2023-11-04T05:29:12.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.065+0000 7f8091589700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f807006af00 0x7f807006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.066+0000 7f8091589700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f807006af00 0x7f807006d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8088007fd0 tx=0x7f8088009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.069+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8080056040 con 0x7f808c0d02d0 2023-11-04T05:29:12.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.271+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f808c074df0 con 0x7f808c0d02d0 2023-11-04T05:29:12.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.272+0000 7f807dffb700 1 -- 172.21.15.3:0/1848279776 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8080059660 con 0x7f808c0d02d0 2023-11-04T05:29:12.273 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:12.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.273+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f807006af00 msgr2=0x7f807006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.273+0000 7f80937ed700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f807006af00 0x7f807006d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8088007fd0 tx=0x7f8088009040 comp rx=0 tx=0).stop 2023-11-04T05:29:12.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 msgr2=0x7f808c0b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 0x7f808c0b31b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f808c0d1750 tx=0x7f8080004750 comp rx=0 tx=0).stop 2023-11-04T05:29:12.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 shutdown_connections 2023-11-04T05:29:12.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f807006af00 0x7f807006d3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06ab20 0x7f808c1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c1611d0 0x7f808c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 --2- 172.21.15.3:0/1848279776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f808c0d02d0 0x7f808c0b31b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 >> 172.21.15.3:0/1848279776 conn(0x7f808c072560 msgr2=0x7f808c0b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:12.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 shutdown_connections 2023-11-04T05:29:12.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.274+0000 7f80937ed700 1 -- 172.21.15.3:0/1848279776 wait complete. 2023-11-04T05:29:12.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.414+0000 7fe72909f700 1 Processor -- start 2023-11-04T05:29:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.414+0000 7fe72909f700 1 -- start start 2023-11-04T05:29:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.414+0000 7fe72909f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe72415db50 0x7fe7240cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.414+0000 7fe72909f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7240d2800 0x7fe7240cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.414+0000 7fe72909f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d0330 0x7fe7240d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72909f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240d0cc0 con 0x7fe7240d2800 2023-11-04T05:29:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72909f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240d1510 con 0x7fe72415db50 2023-11-04T05:29:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72909f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240d1760 con 0x7fe7240d0330 2023-11-04T05:29:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7240d2800 0x7fe7240cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7240d2800 0x7fe7240cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52950/0 (socket says 172.21.15.3:52950) 2023-11-04T05:29:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 -- 172.21.15.3:0/4030940607 learned_addr learned my addr 172.21.15.3:0/4030940607 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe722d9d700 1 --2- 172.21.15.3:0/4030940607 >> v2:172.21.15.188:3300/0 conn(0x7fe72415db50 0x7fe7240cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72359e700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d0330 0x7fe7240d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 -- 172.21.15.3:0/4030940607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d0330 msgr2=0x7fe7240d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d0330 0x7fe7240d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 -- 172.21.15.3:0/4030940607 >> v2:172.21.15.188:3300/0 conn(0x7fe72415db50 msgr2=0x7fe7240cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 --2- 172.21.15.3:0/4030940607 >> v2:172.21.15.188:3300/0 conn(0x7fe72415db50 0x7fe7240cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 -- 172.21.15.3:0/4030940607 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240d1a40 con 0x7fe7240d2800 2023-11-04T05:29:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72359e700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d0330 0x7fe7240d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe72259c700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7240d2800 0x7fe7240cf5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe72406b780 tx=0x7fe718009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3b431b527bb8a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe721d9b700 1 -- 172.21.15.3:0/4030940607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71800b870 con 0x7fe7240d2800 2023-11-04T05:29:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe721d9b700 1 -- 172.21.15.3:0/4030940607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71800b9d0 con 0x7fe7240d2800 2023-11-04T05:29:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.415+0000 7fe721d9b700 1 -- 172.21.15.3:0/4030940607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71800bd60 con 0x7fe7240d2800 2023-11-04T05:29:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 -- 172.21.15.3:0/4030940607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7240d2800 msgr2=0x7fe7240cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7240d2800 0x7fe7240cf5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe72406b780 tx=0x7fe718009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 -- 172.21.15.3:0/4030940607 shutdown_connections 2023-11-04T05:29:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d0330 0x7fe7240d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 --2- 172.21.15.3:0/4030940607 >> v2:172.21.15.188:3300/0 conn(0x7fe72415db50 0x7fe7240cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 --2- 172.21.15.3:0/4030940607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7240d2800 0x7fe7240cf5c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 -- 172.21.15.3:0/4030940607 >> 172.21.15.3:0/4030940607 conn(0x7fe72408aff0 msgr2=0x7fe7240d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 -- 172.21.15.3:0/4030940607 shutdown_connections 2023-11-04T05:29:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.416+0000 7fe72909f700 1 -- 172.21.15.3:0/4030940607 wait complete. 2023-11-04T05:29:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.417+0000 7fe72909f700 1 Processor -- start 2023-11-04T05:29:12.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.417+0000 7fe72909f700 1 -- start start 2023-11-04T05:29:12.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72909f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7240d0330 0x7fe7240b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72909f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d2800 0x7fe7240c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72909f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 0x7fe7240c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72909f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c8640 con 0x7fe72415db50 2023-11-04T05:29:12.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72909f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c79c0 con 0x7fe7240d0330 2023-11-04T05:29:12.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72909f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c7ca0 con 0x7fe7240d2800 2023-11-04T05:29:12.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 0x7fe7240c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 0x7fe7240c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52958/0 (socket says 172.21.15.3:52958) 2023-11-04T05:29:12.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.418+0000 7fe72359e700 1 -- 172.21.15.3:0/3795028757 learned_addr learned my addr 172.21.15.3:0/3795028757 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:12.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72259c700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d2800 0x7fe7240c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe722d9d700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7240d0330 0x7fe7240b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72359e700 1 -- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d2800 msgr2=0x7fe7240c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72359e700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d2800 0x7fe7240c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72359e700 1 -- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7240d0330 msgr2=0x7fe7240b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72359e700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7240d0330 0x7fe7240b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72359e700 1 -- 172.21.15.3:0/3795028757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718009550 con 0x7fe72415db50 2023-11-04T05:29:12.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72359e700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 0x7fe7240c8fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00cef0 tx=0x7fe71c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00ae50 con 0x7fe72415db50 2023-11-04T05:29:12.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c015440 con 0x7fe72415db50 2023-11-04T05:29:12.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7240c7f80 con 0x7fe72415db50 2023-11-04T05:29:12.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0157f0 con 0x7fe72415db50 2023-11-04T05:29:12.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.419+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7240b5010 con 0x7fe72415db50 2023-11-04T05:29:12.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.420+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7240c8780 con 0x7fe72415db50 2023-11-04T05:29:12.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.420+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe71c015950 con 0x7fe72415db50 2023-11-04T05:29:12.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.421+0000 7fe70b7fe700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe71006af00 0x7fe71006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.421+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe71c089f90 con 0x7fe72415db50 2023-11-04T05:29:12.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.421+0000 7fe722d9d700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe71006af00 0x7fe71006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.422+0000 7fe722d9d700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe71006af00 0x7fe71006d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe72415f050 tx=0x7fe714008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.425+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fe71c0bce40 con 0x7fe72415db50 2023-11-04T05:29:12.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.627+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7240d1030 con 0x7fe72415db50 2023-11-04T05:29:12.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.628+0000 7fe70b7fe700 1 -- 172.21.15.3:0/3795028757 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe71c01a020 con 0x7fe72415db50 2023-11-04T05:29:12.629 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe71006af00 msgr2=0x7fe71006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe71006af00 0x7fe71006d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe72415f050 tx=0x7fe714008040 comp rx=0 tx=0).stop 2023-11-04T05:29:12.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 msgr2=0x7fe7240c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 0x7fe7240c8fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00cef0 tx=0x7fe71c00a340 comp rx=0 tx=0).stop 2023-11-04T05:29:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 shutdown_connections 2023-11-04T05:29:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe71006af00 0x7fe71006d3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7240d0330 0x7fe7240b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72415db50 0x7fe7240c8fa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 --2- 172.21.15.3:0/3795028757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7240d2800 0x7fe7240c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 >> 172.21.15.3:0/3795028757 conn(0x7fe72408aff0 msgr2=0x7fe72415cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:12.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 shutdown_connections 2023-11-04T05:29:12.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.630+0000 7fe72909f700 1 -- 172.21.15.3:0/3795028757 wait complete. 2023-11-04T05:29:12.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.771+0000 7fd28b425700 1 Processor -- start 2023-11-04T05:29:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.771+0000 7fd28b425700 1 -- start start 2023-11-04T05:29:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd28b425700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 0x7fd28415f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd28b425700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd284070a90 con 0x7fd28415ba10 2023-11-04T05:29:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd28b425700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840718f0 con 0x7fd2840d0d70 2023-11-04T05:29:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd28b425700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd284070ce0 con 0x7fd2840b3820 2023-11-04T05:29:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd2891c1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd28415ba10 0x7fd2840c51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58310/0 (socket says 172.21.15.3:58310) 2023-11-04T05:29:12.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd2891c1700 1 -- 172.21.15.3:0/3780166382 learned_addr learned my addr 172.21.15.3:0/3780166382 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35272508 0 0) 0x7fd284070a90 con 0x7fd28415ba10 2023-11-04T05:29:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2700035b0 con 0x7fd28415ba10 2023-11-04T05:29:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182751000 0 0) 0x7fd2700035b0 con 0x7fd28415ba10 2023-11-04T05:29:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 msgr2=0x7fd28415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 0x7fd28415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 >> v1:172.21.15.188:6789/0 conn(0x7fd2840d0d70 legacy=0x7fd2840b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd284070fc0 con 0x7fd28415ba10 2023-11-04T05:29:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fd274001670 con 0x7fd28415ba10 2023-11-04T05:29:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 >> v1:172.21.15.3:6789/0 conn(0x7fd28415ba10 legacy=0x7fd2840c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270003a30 0x7fd270003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.772+0000 7fd27bfff700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd270004850 0x7fd270004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 0x7fd2700052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284070ce0 con 0x7fd270004850 2023-11-04T05:29:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27000bc00 con 0x7fd270003a30 2023-11-04T05:29:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27000bdc0 con 0x7fd2840b3820 2023-11-04T05:29:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd270004850 0x7fd270004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2891c1700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270003a30 0x7fd270003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2899c2700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 0x7fd2700052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 -- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 msgr2=0x7fd2700052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2840b3820 0x7fd2700052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 -- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270003a30 msgr2=0x7fd270003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:12.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270003a30 0x7fd270003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 -- 172.21.15.3:0/3780166382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2700035b0 con 0x7fd270004850 2023-11-04T05:29:12.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2891c1700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270003a30 0x7fd270003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:12.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd2889c0700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd270004850 0x7fd270004cd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd2800016c0 tx=0x7fd270005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c9703dd4b019e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800052d0 con 0x7fd270004850 2023-11-04T05:29:12.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280005430 con 0x7fd270004850 2023-11-04T05:29:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.773+0000 7fd27bfff700 1 -- 172.21.15.3:0/3780166382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800038e0 con 0x7fd270004850 2023-11-04T05:29:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd28b425700 1 -- 172.21.15.3:0/3780166382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd270004850 msgr2=0x7fd270004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd28b425700 1 --2- 172.21.15.3:0/3780166382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd270004850 0x7fd270004cd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd2800016c0 tx=0x7fd270005880 comp rx=0 tx=0).stop 2023-11-04T05:29:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd2899c2700 1 -- 172.21.15.3:0/3780166382 reap_dead start 2023-11-04T05:29:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd28b425700 1 -- 172.21.15.3:0/3780166382 shutdown_connections 2023-11-04T05:29:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd28b425700 1 -- 172.21.15.3:0/3780166382 >> 172.21.15.3:0/3780166382 conn(0x7fd2840ce110 msgr2=0x7fd2840ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd28b425700 1 -- 172.21.15.3:0/3780166382 shutdown_connections 2023-11-04T05:29:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.774+0000 7fd28b425700 1 -- 172.21.15.3:0/3780166382 wait complete. 2023-11-04T05:29:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.775+0000 7fd28b425700 1 Processor -- start 2023-11-04T05:29:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.776+0000 7fd28b425700 1 -- start start 2023-11-04T05:29:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.776+0000 7fd28b425700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd270004850 0x7fd28406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.776+0000 7fd28b425700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2841507f0 0x7fd28415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.776+0000 7fd28b425700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 0x7fd28415dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.776+0000 7fd28b425700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28406d4e0 con 0x7fd284150c70 2023-11-04T05:29:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd28b425700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28406cc10 con 0x7fd2841507f0 2023-11-04T05:29:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd28b425700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2841f7790 con 0x7fd270004850 2023-11-04T05:29:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 0x7fd28415dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 0x7fd28415dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52978/0 (socket says 172.21.15.3:52978) 2023-11-04T05:29:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 -- 172.21.15.3:0/3805811261 learned_addr learned my addr 172.21.15.3:0/3805811261 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2889c0700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2841507f0 0x7fd28415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2891c1700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd270004850 0x7fd28406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 -- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd270004850 msgr2=0x7fd28406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd270004850 0x7fd28406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 -- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2841507f0 msgr2=0x7fd28415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2841507f0 0x7fd28415d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 -- 172.21.15.3:0/3805811261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28406ce20 con 0x7fd284150c70 2023-11-04T05:29:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd2899c2700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 0x7fd28415dd00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd284068840 tx=0x7fd27c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c003c00 con 0x7fd284150c70 2023-11-04T05:29:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c012070 con 0x7fd284150c70 2023-11-04T05:29:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28406d100 con 0x7fd284150c70 2023-11-04T05:29:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.777+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00e690 con 0x7fd284150c70 2023-11-04T05:29:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.778+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2841f6c60 con 0x7fd284150c70 2023-11-04T05:29:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.779+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28406d620 con 0x7fd284150c70 2023-11-04T05:29:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.779+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd27c00e7f0 con 0x7fd284150c70 2023-11-04T05:29:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.779+0000 7fd279ffb700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd26806af00 0x7fd26806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.779+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd27c088fd0 con 0x7fd284150c70 2023-11-04T05:29:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.780+0000 7fd2891c1700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd26806af00 0x7fd26806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.780+0000 7fd2891c1700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd26806af00 0x7fd26806d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd274000c00 tx=0x7fd274007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.784+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd28406d620 con 0x7fd284150c70 2023-11-04T05:29:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.989+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd28406d620 con 0x7fd284150c70 2023-11-04T05:29:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.989+0000 7fd279ffb700 1 -- 172.21.15.3:0/3805811261 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd28406d620 con 0x7fd284150c70 2023-11-04T05:29:12.991 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd26806af00 msgr2=0x7fd26806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd26806af00 0x7fd26806d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd274000c00 tx=0x7fd274007040 comp rx=0 tx=0).stop 2023-11-04T05:29:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 msgr2=0x7fd28415dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 0x7fd28415dd00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd284068840 tx=0x7fd27c00dab0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 shutdown_connections 2023-11-04T05:29:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd26806af00 0x7fd26806d3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd284150c70 0x7fd28415dd00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd270004850 0x7fd28406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 --2- 172.21.15.3:0/3805811261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2841507f0 0x7fd28415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 >> 172.21.15.3:0/3805811261 conn(0x7fd2840ce110 msgr2=0x7fd2841e5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 shutdown_connections 2023-11-04T05:29:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:12.991+0000 7fd28b425700 1 -- 172.21.15.3:0/3805811261 wait complete. 2023-11-04T05:29:13.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:13.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.131+0000 7f66bc473700 1 Processor -- start 2023-11-04T05:29:13.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.131+0000 7f66bc473700 1 -- start start 2023-11-04T05:29:13.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.132+0000 7f66bc473700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 0x7f66b415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.132+0000 7f66bc473700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b40ccc70 con 0x7f66b40ceaa0 2023-11-04T05:29:13.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.132+0000 7f66bc473700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b415baa0 con 0x7f66b415b260 2023-11-04T05:29:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.132+0000 7f66bc473700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40ccec0 con 0x7f66b40cee80 2023-11-04T05:29:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.132+0000 7f66ba20f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66b40ceaa0 0x7f66b40cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58330/0 (socket says 172.21.15.3:58330) 2023-11-04T05:29:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.132+0000 7f66ba20f700 1 -- 172.21.15.3:0/2259965443 learned_addr learned my addr 172.21.15.3:0/2259965443 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b9a0e700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 0x7f66b415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147236744 0 0) 0x7f66b40ccc70 con 0x7f66b40ceaa0 2023-11-04T05:29:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a00035b0 con 0x7f66b40ceaa0 2023-11-04T05:29:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 494074248 0 0) 0x7f66a00035b0 con 0x7f66b40ceaa0 2023-11-04T05:29:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 msgr2=0x7f66b415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 0x7f66b415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 >> v1:172.21.15.188:6789/0 conn(0x7f66b415b260 legacy=0x7f66b40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40cd1a0 con 0x7f66b40ceaa0 2023-11-04T05:29:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f66a4001670 con 0x7f66b40ceaa0 2023-11-04T05:29:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 >> v1:172.21.15.3:6789/0 conn(0x7f66b40ceaa0 legacy=0x7f66b40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 0x7f66a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0004d90 0x7f66a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40ccec0 con 0x7f66a0003a30 2023-11-04T05:29:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a000bac0 con 0x7f66a0004d90 2023-11-04T05:29:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a000bc80 con 0x7f66b40cee80 2023-11-04T05:29:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66ba20f700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66b9a0e700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 0x7f66a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66baa10700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0004d90 0x7f66a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.133+0000 7f66ba20f700 1 -- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 msgr2=0x7f66a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66ba20f700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40cee80 0x7f66a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66ba20f700 1 -- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0004d90 msgr2=0x7f66a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66ba20f700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0004d90 0x7f66a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66ba20f700 1 -- 172.21.15.3:0/2259965443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00035b0 con 0x7f66a0003a30 2023-11-04T05:29:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66baa10700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0004d90 0x7f66a0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66ba20f700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66a0003e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c240 tx=0x7f66a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8797bfa9d8f37a1f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400cde0 con 0x7f66a0003a30 2023-11-04T05:29:13.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a4007ba0 con 0x7f66a0003a30 2023-11-04T05:29:13.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66b920d700 1 -- 172.21.15.3:0/2259965443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4005ba0 con 0x7f66a0003a30 2023-11-04T05:29:13.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66bc473700 1 -- 172.21.15.3:0/2259965443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 msgr2=0x7f66a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66bc473700 1 --2- 172.21.15.3:0/2259965443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66a0003e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c240 tx=0x7f66a0005880 comp rx=0 tx=0).stop 2023-11-04T05:29:13.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66baa10700 1 -- 172.21.15.3:0/2259965443 reap_dead start 2023-11-04T05:29:13.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66bc473700 1 -- 172.21.15.3:0/2259965443 shutdown_connections 2023-11-04T05:29:13.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66bc473700 1 -- 172.21.15.3:0/2259965443 >> 172.21.15.3:0/2259965443 conn(0x7f66b4072560 msgr2=0x7f66b4099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:13.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66bc473700 1 -- 172.21.15.3:0/2259965443 shutdown_connections 2023-11-04T05:29:13.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.134+0000 7f66bc473700 1 -- 172.21.15.3:0/2259965443 wait complete. 2023-11-04T05:29:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.136+0000 7f66bc473700 1 Processor -- start 2023-11-04T05:29:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.136+0000 7f66bc473700 1 -- start start 2023-11-04T05:29:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.136+0000 7f66bc473700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66b4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.136+0000 7f66bc473700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b41554a0 0x7f66b40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.136+0000 7f66bc473700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b1500 0x7f66b40ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.136+0000 7f66bc473700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4155940 con 0x7f66a0003a30 2023-11-04T05:29:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66bc473700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40b0020 con 0x7f66b41554a0 2023-11-04T05:29:13.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66bc473700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40b0300 con 0x7f66b40b1500 2023-11-04T05:29:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66b4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66b4154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52994/0 (socket says 172.21.15.3:52994) 2023-11-04T05:29:13.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 -- 172.21.15.3:0/802248712 learned_addr learned my addr 172.21.15.3:0/802248712 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:13.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66b9a0e700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b41554a0 0x7f66b40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66baa10700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b1500 0x7f66b40ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 -- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b1500 msgr2=0x7f66b40ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b1500 0x7f66b40ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 -- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b41554a0 msgr2=0x7f66b40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b41554a0 0x7f66b40ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 -- 172.21.15.3:0/802248712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40b05e0 con 0x7f66a0003a30 2023-11-04T05:29:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66b9a0e700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b41554a0 0x7f66b40ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:13.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66ba20f700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66b4154f60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f66a4007d50 tx=0x7f66b40aff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a40070d0 con 0x7f66a0003a30 2023-11-04T05:29:13.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a4007230 con 0x7f66a0003a30 2023-11-04T05:29:13.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a40075e0 con 0x7f66a0003a30 2023-11-04T05:29:13.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40b08c0 con 0x7f66a0003a30 2023-11-04T05:29:13.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.137+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b40ca220 con 0x7f66a0003a30 2023-11-04T05:29:13.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.138+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b40b1ab0 con 0x7f66a0003a30 2023-11-04T05:29:13.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.139+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66a4019030 con 0x7f66a0003a30 2023-11-04T05:29:13.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.139+0000 7f66aaffd700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669806af00 0x7f669806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.139+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f66a408a7b0 con 0x7f66a0003a30 2023-11-04T05:29:13.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.139+0000 7f66b9a0e700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669806af00 0x7f669806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.140+0000 7f66b9a0e700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669806af00 0x7f669806d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66b40b1a40 tx=0x7f66b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.144+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f66a40555f0 con 0x7f66a0003a30 2023-11-04T05:29:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.347+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f66b4155940 con 0x7f66a0003a30 2023-11-04T05:29:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.347+0000 7f66aaffd700 1 -- 172.21.15.3:0/802248712 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f66a4058c10 con 0x7f66a0003a30 2023-11-04T05:29:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669806af00 msgr2=0x7f669806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669806af00 0x7f669806d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66b40b1a40 tx=0x7f66b0008040 comp rx=0 tx=0).stop 2023-11-04T05:29:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 msgr2=0x7f66b4154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66b4154f60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f66a4007d50 tx=0x7f66b40aff10 comp rx=0 tx=0).stop 2023-11-04T05:29:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 shutdown_connections 2023-11-04T05:29:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f669806af00 0x7f669806d3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b40b1500 0x7f66b40ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66a0003a30 0x7f66b4154f60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 --2- 172.21.15.3:0/802248712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b41554a0 0x7f66b40ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 >> 172.21.15.3:0/802248712 conn(0x7f66b4072560 msgr2=0x7f66b415aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 shutdown_connections 2023-11-04T05:29:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.349+0000 7f66bc473700 1 -- 172.21.15.3:0/802248712 wait complete. 2023-11-04T05:29:13.387 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-11-04T05:29:13.388 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.512+0000 7ff870f8a700 1 Processor -- start 2023-11-04T05:29:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.512+0000 7ff870f8a700 1 -- start start 2023-11-04T05:29:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff870f8a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 0x7ff86c06af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff870f8a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c0d04d0 con 0x7ff86c0cdc80 2023-11-04T05:29:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff870f8a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c0b3aa0 con 0x7ff86c15bb00 2023-11-04T05:29:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff870f8a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0cab90 con 0x7ff86c06ab20 2023-11-04T05:29:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff86ad9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff86c0cdc80 0x7ff86c0b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58354/0 (socket says 172.21.15.3:58354) 2023-11-04T05:29:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff86ad9d700 1 -- 172.21.15.3:0/2342877277 learned_addr learned my addr 172.21.15.3:0/2342877277 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735245664 0 0) 0x7ff86c0d04d0 con 0x7ff86c0cdc80 2023-11-04T05:29:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.513+0000 7ff86a59c700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 0x7ff86c06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8540035b0 con 0x7ff86c0cdc80 2023-11-04T05:29:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952336610 0 0) 0x7ff8540035b0 con 0x7ff86c0cdc80 2023-11-04T05:29:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 msgr2=0x7ff86c06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 0x7ff86c06af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 >> v1:172.21.15.188:6789/0 conn(0x7ff86c15bb00 legacy=0x7ff86c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c06b440 con 0x7ff86c0cdc80 2023-11-04T05:29:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7ff8640019f0 con 0x7ff86c0cdc80 2023-11-04T05:29:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 >> v1:172.21.15.3:6789/0 conn(0x7ff86c0cdc80 legacy=0x7ff86c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854003a30 0x7ff854003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 0x7ff854004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff854004d90 0x7ff854005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0cab90 con 0x7ff854004d90 2023-11-04T05:29:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85400bac0 con 0x7ff854003a30 2023-11-04T05:29:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85400bc80 con 0x7ff86c06ab20 2023-11-04T05:29:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff854004d90 0x7ff854005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86a59c700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854003a30 0x7ff854003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff869d9b700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 0x7ff854004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86ad9d700 1 -- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 msgr2=0x7ff854004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c06ab20 0x7ff854004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86ad9d700 1 -- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854003a30 msgr2=0x7ff854003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854003a30 0x7ff854003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.514+0000 7ff86ad9d700 1 -- 172.21.15.3:0/2342877277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540035b0 con 0x7ff854004d90 2023-11-04T05:29:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff854004d90 0x7ff854005210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff86c06cc80 tx=0x7ff854005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23708832d439ddbf server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86400a2f0 con 0x7ff854004d90 2023-11-04T05:29:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff864012600 con 0x7ff854004d90 2023-11-04T05:29:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff86959a700 1 -- 172.21.15.3:0/2342877277 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640129b0 con 0x7ff854004d90 2023-11-04T05:29:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff870f8a700 1 -- 172.21.15.3:0/2342877277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff854004d90 msgr2=0x7ff854005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff870f8a700 1 --2- 172.21.15.3:0/2342877277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff854004d90 0x7ff854005210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff86c06cc80 tx=0x7ff854005880 comp rx=0 tx=0).stop 2023-11-04T05:29:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff86ad9d700 1 -- 172.21.15.3:0/2342877277 reap_dead start 2023-11-04T05:29:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff870f8a700 1 -- 172.21.15.3:0/2342877277 shutdown_connections 2023-11-04T05:29:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff870f8a700 1 -- 172.21.15.3:0/2342877277 >> 172.21.15.3:0/2342877277 conn(0x7ff86c06df90 msgr2=0x7ff86c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff870f8a700 1 -- 172.21.15.3:0/2342877277 shutdown_connections 2023-11-04T05:29:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.515+0000 7ff870f8a700 1 -- 172.21.15.3:0/2342877277 wait complete. 2023-11-04T05:29:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.517+0000 7ff870f8a700 1 Processor -- start 2023-11-04T05:29:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.517+0000 7ff870f8a700 1 -- start start 2023-11-04T05:29:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.517+0000 7ff870f8a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854004d90 0x7ff86c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.517+0000 7ff870f8a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c1f51a0 0x7ff86c0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.517+0000 7ff870f8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 0x7ff86c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.517+0000 7ff870f8a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0c8020 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff870f8a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0ca480 con 0x7ff854004d90 2023-11-04T05:29:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff870f8a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c1f5620 con 0x7ff86c1f51a0 2023-11-04T05:29:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 0x7ff86c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 0x7ff86c0c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53018/0 (socket says 172.21.15.3:53018) 2023-11-04T05:29:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 -- 172.21.15.3:0/1294064938 learned_addr learned my addr 172.21.15.3:0/1294064938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86a59c700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854004d90 0x7ff86c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff869d9b700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c1f51a0 0x7ff86c0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 -- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c1f51a0 msgr2=0x7ff86c0c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c1f51a0 0x7ff86c0c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 -- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854004d90 msgr2=0x7ff86c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854004d90 0x7ff86c159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 -- 172.21.15.3:0/1294064938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c0c8d90 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86a59c700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854004d90 0x7ff86c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff86ad9d700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 0x7ff86c0c9f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff864000f80 tx=0x7ff86c0cb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640017b0 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff864001910 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86401e650 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c0c8210 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.518+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c0c87d0 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.519+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c0c4100 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.520+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff86401e7b0 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.520+0000 7ff85affd700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff84c06af30 0x7ff84c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.520+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff86408b7f0 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.520+0000 7ff86a59c700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff84c06af30 0x7ff84c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.521+0000 7ff86a59c700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff84c06af30 0x7ff84c06d3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff860009c60 tx=0x7ff860008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.525+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff86c0c4100 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.726+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff86c0c4100 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.728+0000 7ff85affd700 1 -- 172.21.15.3:0/1294064938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7ff86c0c4100 con 0x7ff86c0c9ac0 2023-11-04T05:29:13.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:13.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff84c06af30 msgr2=0x7ff84c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff84c06af30 0x7ff84c06d3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff860009c60 tx=0x7ff860008040 comp rx=0 tx=0).stop 2023-11-04T05:29:13.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 msgr2=0x7ff86c0c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 0x7ff86c0c9f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff864000f80 tx=0x7ff86c0cb920 comp rx=0 tx=0).stop 2023-11-04T05:29:13.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 shutdown_connections 2023-11-04T05:29:13.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff84c06af30 0x7ff84c06d3f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff854004d90 0x7ff86c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff86c0c9ac0 0x7ff86c0c9f40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 --2- 172.21.15.3:0/1294064938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86c1f51a0 0x7ff86c0c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 >> 172.21.15.3:0/1294064938 conn(0x7ff86c06df90 msgr2=0x7ff86c0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:13.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 shutdown_connections 2023-11-04T05:29:13.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.730+0000 7ff870f8a700 1 -- 172.21.15.3:0/1294064938 wait complete. 2023-11-04T05:29:13.743 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:13.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:13.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.874+0000 7fbc5ff37700 1 Processor -- start 2023-11-04T05:29:13.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.874+0000 7fbc5ff37700 1 -- start start 2023-11-04T05:29:13.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ff37700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc580cee80 0x7fbc5815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ff37700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc5815b260 0x7fbc580caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ff37700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc580ccc70 con 0x7fbc580ceaa0 2023-11-04T05:29:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ff37700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5815baa0 con 0x7fbc5815b260 2023-11-04T05:29:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ff37700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc580ccec0 con 0x7fbc580cee80 2023-11-04T05:29:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc5815b260 0x7fbc580caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbc5815b260 0x7fbc580caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53026/0 (socket says 172.21.15.3:53026) 2023-11-04T05:29:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 -- 172.21.15.3:0/2899278822 learned_addr learned my addr 172.21.15.3:0/2899278822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 -- 172.21.15.3:0/2899278822 >> v2:172.21.15.188:3300/0 conn(0x7fbc580cee80 msgr2=0x7fbc5815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 --2- 172.21.15.3:0/2899278822 >> v2:172.21.15.188:3300/0 conn(0x7fbc580cee80 0x7fbc5815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 -- 172.21.15.3:0/2899278822 >> v1:172.21.15.188:6789/0 conn(0x7fbc580ceaa0 legacy=0x7fbc580cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 -- 172.21.15.3:0/2899278822 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580cd1a0 con 0x7fbc5815b260 2023-11-04T05:29:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5e4d4700 1 --2- 172.21.15.3:0/2899278822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5815b260 0x7fbc580caff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbc5806dfe0 tx=0x7fbc5000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8751a449e4adb1bd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ccd1700 1 -- 172.21.15.3:0/2899278822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50010070 con 0x7fbc5815b260 2023-11-04T05:29:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ccd1700 1 -- 172.21.15.3:0/2899278822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5000dd90 con 0x7fbc5815b260 2023-11-04T05:29:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.875+0000 7fbc5ccd1700 1 -- 172.21.15.3:0/2899278822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5000b310 con 0x7fbc5815b260 2023-11-04T05:29:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2899278822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5815b260 msgr2=0x7fbc580caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2899278822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5815b260 0x7fbc580caff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbc5806dfe0 tx=0x7fbc5000d430 comp rx=0 tx=0).stop 2023-11-04T05:29:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2899278822 shutdown_connections 2023-11-04T05:29:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2899278822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc5815b260 0x7fbc580caff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2899278822 >> v2:172.21.15.188:3300/0 conn(0x7fbc580cee80 0x7fbc5815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2899278822 >> 172.21.15.3:0/2899278822 conn(0x7fbc58072560 msgr2=0x7fbc58099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2899278822 shutdown_connections 2023-11-04T05:29:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.876+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2899278822 wait complete. 2023-11-04T05:29:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.877+0000 7fbc5ff37700 1 Processor -- start 2023-11-04T05:29:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.877+0000 7fbc5ff37700 1 -- start start 2023-11-04T05:29:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5ff37700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 0x7fbc58150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5ff37700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc580cee80 0x7fbc58154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5ff37700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc5815b260 0x7fbc580b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5ff37700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc581521d0 con 0x7fbc580ceaa0 2023-11-04T05:29:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5ff37700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc581584f0 con 0x7fbc5815b260 2023-11-04T05:29:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5ff37700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc581523c0 con 0x7fbc580cee80 2023-11-04T05:29:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 0x7fbc58150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 0x7fbc58150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53028/0 (socket says 172.21.15.3:53028) 2023-11-04T05:29:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 -- 172.21.15.3:0/2927626930 learned_addr learned my addr 172.21.15.3:0/2927626930 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5d4d2700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc580cee80 0x7fbc58154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5e4d4700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc5815b260 0x7fbc580b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 -- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc580cee80 msgr2=0x7fbc58154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc580cee80 0x7fbc58154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 -- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc5815b260 msgr2=0x7fbc580b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc5815b260 0x7fbc580b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.878+0000 7fbc5dcd3700 1 -- 172.21.15.3:0/2927626930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5000bd40 con 0x7fbc580ceaa0 2023-11-04T05:29:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.879+0000 7fbc5dcd3700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 0x7fbc58150e80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c4e0 tx=0x7fbc5400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.879+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540052d0 con 0x7fbc580ceaa0 2023-11-04T05:29:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.879+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc54005430 con 0x7fbc580ceaa0 2023-11-04T05:29:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.879+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540038c0 con 0x7fbc580ceaa0 2023-11-04T05:29:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.879+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc581526a0 con 0x7fbc580ceaa0 2023-11-04T05:29:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.879+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc58158740 con 0x7fbc580ceaa0 2023-11-04T05:29:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.880+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc58158960 con 0x7fbc580ceaa0 2023-11-04T05:29:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.880+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc54003a20 con 0x7fbc580ceaa0 2023-11-04T05:29:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.881+0000 7fbc4e7fc700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbc3c06af00 0x7fbc3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.881+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbc54088e50 con 0x7fbc580ceaa0 2023-11-04T05:29:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.881+0000 7fbc5d4d2700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbc3c06af00 0x7fbc3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.881+0000 7fbc5d4d2700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbc3c06af00 0x7fbc3c06d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc580d1730 tx=0x7fbc48009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:13.885+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fbc54053a10 con 0x7fbc580ceaa0 2023-11-04T05:29:14.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.087+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc580d1480 con 0x7fbc580ceaa0 2023-11-04T05:29:14.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.087+0000 7fbc4e7fc700 1 -- 172.21.15.3:0/2927626930 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc54017020 con 0x7fbc580ceaa0 2023-11-04T05:29:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:14.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbc3c06af00 msgr2=0x7fbc3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbc3c06af00 0x7fbc3c06d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc580d1730 tx=0x7fbc48009190 comp rx=0 tx=0).stop 2023-11-04T05:29:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 msgr2=0x7fbc58150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 0x7fbc58150e80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c4e0 tx=0x7fbc5400c7f0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 shutdown_connections 2023-11-04T05:29:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fbc3c06af00 0x7fbc3c06d3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc5815b260 0x7fbc580b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc580ceaa0 0x7fbc58150e80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 --2- 172.21.15.3:0/2927626930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc580cee80 0x7fbc58154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 >> 172.21.15.3:0/2927626930 conn(0x7fbc58072560 msgr2=0x7fbc5806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 shutdown_connections 2023-11-04T05:29:14.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.089+0000 7fbc5ff37700 1 -- 172.21.15.3:0/2927626930 wait complete. 2023-11-04T05:29:14.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:14.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 Processor -- start 2023-11-04T05:29:14.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 -- start start 2023-11-04T05:29:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe00cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effe006e2f0 0x7effe00668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7effe00b2f30 0x7effe0066e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe015e1d0 con 0x7effe006e2f0 2023-11-04T05:29:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00cdce0 con 0x7effe00b2f30 2023-11-04T05:29:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.230+0000 7effe655a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cdfc0 con 0x7effe015ddf0 2023-11-04T05:29:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effe006e2f0 0x7effe00668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effe006e2f0 0x7effe00668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53038/0 (socket says 172.21.15.3:53038) 2023-11-04T05:29:14.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 -- 172.21.15.3:0/1269378925 learned_addr learned my addr 172.21.15.3:0/1269378925 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:14.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effe4af7700 1 --2- 172.21.15.3:0/1269378925 >> v2:172.21.15.188:3300/0 conn(0x7effe00b2f30 0x7effe0066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 -- 172.21.15.3:0/1269378925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 msgr2=0x7effe00cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:14.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 --2- 172.21.15.3:0/1269378925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe00cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 -- 172.21.15.3:0/1269378925 >> v2:172.21.15.188:3300/0 conn(0x7effe00b2f30 msgr2=0x7effe0066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:14.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 --2- 172.21.15.3:0/1269378925 >> v2:172.21.15.188:3300/0 conn(0x7effe00b2f30 0x7effe0066e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 -- 172.21.15.3:0/1269378925 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00ce2a0 con 0x7effe006e2f0 2023-11-04T05:29:14.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdf7fe700 1 --2- 172.21.15.3:0/1269378925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe006e2f0 0x7effe00668e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7effe00cd9a0 tx=0x7effd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349fd271fee3e43b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdeffd700 1 -- 172.21.15.3:0/1269378925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400b870 con 0x7effe006e2f0 2023-11-04T05:29:14.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdeffd700 1 -- 172.21.15.3:0/1269378925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400b9d0 con 0x7effe006e2f0 2023-11-04T05:29:14.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.231+0000 7effdeffd700 1 -- 172.21.15.3:0/1269378925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400bd60 con 0x7effe006e2f0 2023-11-04T05:29:14.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 -- 172.21.15.3:0/1269378925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe006e2f0 msgr2=0x7effe00668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 --2- 172.21.15.3:0/1269378925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe006e2f0 0x7effe00668e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7effe00cd9a0 tx=0x7effd4009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 -- 172.21.15.3:0/1269378925 shutdown_connections 2023-11-04T05:29:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 --2- 172.21.15.3:0/1269378925 >> v2:172.21.15.188:3300/0 conn(0x7effe00b2f30 0x7effe0066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 --2- 172.21.15.3:0/1269378925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe00cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 --2- 172.21.15.3:0/1269378925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe006e2f0 0x7effe00668e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 -- 172.21.15.3:0/1269378925 >> 172.21.15.3:0/1269378925 conn(0x7effe00cc860 msgr2=0x7effe00ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:14.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 -- 172.21.15.3:0/1269378925 shutdown_connections 2023-11-04T05:29:14.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.232+0000 7effe655a700 1 -- 172.21.15.3:0/1269378925 wait complete. 2023-11-04T05:29:14.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.233+0000 7effe655a700 1 Processor -- start 2023-11-04T05:29:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.233+0000 7effe655a700 1 -- start start 2023-11-04T05:29:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe655a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe006e2f0 0x7effe01f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe655a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 0x7effe0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe655a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe0146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe655a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe015ccc0 con 0x7effe00b2f30 2023-11-04T05:29:14.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe655a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe015bdf0 con 0x7effe006e2f0 2023-11-04T05:29:14.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe655a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe01f4ae0 con 0x7effe015ddf0 2023-11-04T05:29:14.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 0x7effe0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 0x7effe0157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53044/0 (socket says 172.21.15.3:53044) 2023-11-04T05:29:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 -- 172.21.15.3:0/2364377947 learned_addr learned my addr 172.21.15.3:0/2364377947 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 -- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 msgr2=0x7effe0146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe4af7700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe0146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdffff700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe006e2f0 0x7effe01f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe0146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 -- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe006e2f0 msgr2=0x7effe01f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe006e2f0 0x7effe01f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdf7fe700 1 -- 172.21.15.3:0/2364377947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4009550 con 0x7effe00b2f30 2023-11-04T05:29:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effe4af7700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.234+0000 7effdffff700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe006e2f0 0x7effe01f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.235+0000 7effdf7fe700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 0x7effe0157df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7effe006d610 tx=0x7effd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.235+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4004310 con 0x7effe00b2f30 2023-11-04T05:29:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.235+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd401f070 con 0x7effe00b2f30 2023-11-04T05:29:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.235+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe015c040 con 0x7effe00b2f30 2023-11-04T05:29:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.235+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40036a0 con 0x7effe00b2f30 2023-11-04T05:29:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.235+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe015c600 con 0x7effe00b2f30 2023-11-04T05:29:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.236+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe015ce00 con 0x7effe00b2f30 2023-11-04T05:29:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.236+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effd4021030 con 0x7effe00b2f30 2023-11-04T05:29:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.237+0000 7effdcff9700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7effcc06af00 0x7effcc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.237+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7effd408ae00 con 0x7effe00b2f30 2023-11-04T05:29:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.237+0000 7effdffff700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7effcc06af00 0x7effcc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.237+0000 7effdffff700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7effcc06af00 0x7effcc06d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effd000a8f0 tx=0x7effd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.241+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7effd4055b80 con 0x7effe00b2f30 2023-11-04T05:29:14.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.442+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effe0074df0 con 0x7effe00b2f30 2023-11-04T05:29:14.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.442+0000 7effdcff9700 1 -- 172.21.15.3:0/2364377947 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7effd40591a0 con 0x7effe00b2f30 2023-11-04T05:29:14.444 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:14.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7effcc06af00 msgr2=0x7effcc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7effcc06af00 0x7effcc06d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effd000a8f0 tx=0x7effd0008040 comp rx=0 tx=0).stop 2023-11-04T05:29:14.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 msgr2=0x7effe0157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 0x7effe0157df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7effe006d610 tx=0x7effd4004750 comp rx=0 tx=0).stop 2023-11-04T05:29:14.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 shutdown_connections 2023-11-04T05:29:14.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7effcc06af00 0x7effcc06d3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe006e2f0 0x7effe01f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effe015ddf0 0x7effe0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 --2- 172.21.15.3:0/2364377947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe00b2f30 0x7effe0157df0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 >> 172.21.15.3:0/2364377947 conn(0x7effe00cc860 msgr2=0x7effe009cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:14.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 shutdown_connections 2023-11-04T05:29:14.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.444+0000 7effe655a700 1 -- 172.21.15.3:0/2364377947 wait complete. 2023-11-04T05:29:14.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:14.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.586+0000 7f797ac9a700 1 Processor -- start 2023-11-04T05:29:14.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.586+0000 7f797ac9a700 1 -- start start 2023-11-04T05:29:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f797ac9a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f797406ab20 0x7f797406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f797ac9a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740caca0 con 0x7f79740d02d0 2023-11-04T05:29:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f797ac9a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740cfe40 con 0x7f79741611d0 2023-11-04T05:29:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f797ac9a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79740ce0a0 con 0x7f797406ab20 2023-11-04T05:29:14.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f7973fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f797406ab20 0x7f797406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f7979237700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79740d02d0 0x7f79740d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58402/0 (socket says 172.21.15.3:58402) 2023-11-04T05:29:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f7979237700 1 -- 172.21.15.3:0/3409836159 learned_addr learned my addr 172.21.15.3:0/3409836159 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f7973fff700 1 -- 172.21.15.3:0/3409836159 >> v1:172.21.15.188:6789/0 conn(0x7f79741611d0 legacy=0x7f79740ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f7973fff700 1 -- 172.21.15.3:0/3409836159 >> v1:172.21.15.3:6789/0 conn(0x7f79740d02d0 legacy=0x7f79740d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f7973fff700 1 -- 172.21.15.3:0/3409836159 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740cf850 con 0x7f797406ab20 2023-11-04T05:29:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.587+0000 7f79737fe700 1 -- 172.21.15.3:0/3409836159 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722829585 0 0) 0x7f79740caca0 con 0x7f79740d02d0 2023-11-04T05:29:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f7973fff700 1 --2- 172.21.15.3:0/3409836159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797406ab20 0x7f797406afa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7960009840 tx=0x7f7960009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ba681f992c8731 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f79737fe700 1 -- 172.21.15.3:0/3409836159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960011830 con 0x7f797406ab20 2023-11-04T05:29:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f79737fe700 1 -- 172.21.15.3:0/3409836159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7960011990 con 0x7f797406ab20 2023-11-04T05:29:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f79737fe700 1 -- 172.21.15.3:0/3409836159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960011d20 con 0x7f797406ab20 2023-11-04T05:29:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 -- 172.21.15.3:0/3409836159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797406ab20 msgr2=0x7f797406afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 --2- 172.21.15.3:0/3409836159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797406ab20 0x7f797406afa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7960009840 tx=0x7f7960009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 -- 172.21.15.3:0/3409836159 shutdown_connections 2023-11-04T05:29:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 --2- 172.21.15.3:0/3409836159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797406ab20 0x7f797406afa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 -- 172.21.15.3:0/3409836159 >> 172.21.15.3:0/3409836159 conn(0x7f7974072560 msgr2=0x7f7974099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 -- 172.21.15.3:0/3409836159 shutdown_connections 2023-11-04T05:29:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.588+0000 7f797ac9a700 1 -- 172.21.15.3:0/3409836159 wait complete. 2023-11-04T05:29:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.589+0000 7f797ac9a700 1 Processor -- start 2023-11-04T05:29:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 -- start start 2023-11-04T05:29:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797406ab20 0x7f79741538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79740d02d0 0x7f797415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 0x7f79740ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974160350 con 0x7f79741611d0 2023-11-04T05:29:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79740c4880 con 0x7f79740d02d0 2023-11-04T05:29:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.590+0000 7f797ac9a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797415f7d0 con 0x7f797406ab20 2023-11-04T05:29:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 0x7f79740ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 0x7f79740ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53060/0 (socket says 172.21.15.3:53060) 2023-11-04T05:29:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 -- 172.21.15.3:0/703264916 learned_addr learned my addr 172.21.15.3:0/703264916 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7973fff700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79740d02d0 0x7f797415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7978a36700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797406ab20 0x7f79741538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 -- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797406ab20 msgr2=0x7f79741538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797406ab20 0x7f79741538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 -- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79740d02d0 msgr2=0x7f797415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79740d02d0 0x7f797415eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 -- 172.21.15.3:0/703264916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600094a0 con 0x7f79741611d0 2023-11-04T05:29:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7978a36700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797406ab20 0x7f79741538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f7979237700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 0x7f79740ada30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f796c002230 tx=0x7f796c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c00ac20 con 0x7f79741611d0 2023-11-04T05:29:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c00ada0 con 0x7f79741611d0 2023-11-04T05:29:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797415fab0 con 0x7f79741611d0 2023-11-04T05:29:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c007310 con 0x7f79741611d0 2023-11-04T05:29:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.591+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7974160070 con 0x7f79741611d0 2023-11-04T05:29:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.592+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79740c4a40 con 0x7f79741611d0 2023-11-04T05:29:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.593+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f796c007470 con 0x7f79741611d0 2023-11-04T05:29:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.593+0000 7f79717fa700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f795806af00 0x7f795806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.593+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f796c089880 con 0x7f79741611d0 2023-11-04T05:29:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.593+0000 7f7978a36700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f795806af00 0x7f795806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.594+0000 7f7978a36700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f795806af00 0x7f795806d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7968000f60 tx=0x7f7968009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.598+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f796c054570 con 0x7f79741611d0 2023-11-04T05:29:14.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.800+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7974153e30 con 0x7f79741611d0 2023-11-04T05:29:14.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.800+0000 7f79717fa700 1 -- 172.21.15.3:0/703264916 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f796c0180a0 con 0x7f79741611d0 2023-11-04T05:29:14.801 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:14.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f795806af00 msgr2=0x7f795806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f795806af00 0x7f795806d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7968000f60 tx=0x7f7968009040 comp rx=0 tx=0).stop 2023-11-04T05:29:14.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 msgr2=0x7f79740ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 0x7f79740ada30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f796c002230 tx=0x7f796c00a060 comp rx=0 tx=0).stop 2023-11-04T05:29:14.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 shutdown_connections 2023-11-04T05:29:14.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f795806af00 0x7f795806d3c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79741611d0 0x7f79740ada30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797406ab20 0x7f79741538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 --2- 172.21.15.3:0/703264916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79740d02d0 0x7f797415eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 >> 172.21.15.3:0/703264916 conn(0x7f7974072560 msgr2=0x7f79740b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:14.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 shutdown_connections 2023-11-04T05:29:14.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.802+0000 7f797ac9a700 1 -- 172.21.15.3:0/703264916 wait complete. 2023-11-04T05:29:14.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.942+0000 7fa867d29700 1 Processor -- start 2023-11-04T05:29:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.943+0000 7fa867d29700 1 -- start start 2023-11-04T05:29:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.943+0000 7fa867d29700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8600ceaa0 0x7fa8600cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.943+0000 7fa867d29700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 0x7fa86015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.943+0000 7fa867d29700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600ccc70 con 0x7fa86015b260 2023-11-04T05:29:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.943+0000 7fa867d29700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8600cdee0 con 0x7fa8600ceaa0 2023-11-04T05:29:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.943+0000 7fa867d29700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86015baa0 con 0x7fa8600cee80 2023-11-04T05:29:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa8662c6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa86015b260 0x7fa8600caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58416/0 (socket says 172.21.15.3:58416) 2023-11-04T05:29:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa8662c6700 1 -- 172.21.15.3:0/3594992658 learned_addr learned my addr 172.21.15.3:0/3594992658 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa865ac5700 1 --2- 172.21.15.3:0/3594992658 >> v2:172.21.15.188:3300/0 conn(0x7fa8600ceaa0 0x7fa8600cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa8652c4700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 0x7fa86015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292615837 0 0) 0x7fa8600ccc70 con 0x7fa86015b260 2023-11-04T05:29:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c0035b0 con 0x7fa86015b260 2023-11-04T05:29:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224578213 0 0) 0x7fa84c0035b0 con 0x7fa86015b260 2023-11-04T05:29:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 msgr2=0x7fa86015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 0x7fa86015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 >> v2:172.21.15.188:3300/0 conn(0x7fa8600ceaa0 msgr2=0x7fa8600cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 --2- 172.21.15.3:0/3594992658 >> v2:172.21.15.188:3300/0 conn(0x7fa8600ceaa0 0x7fa8600cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600ccec0 con 0x7fa86015b260 2023-11-04T05:29:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fa8580019f0 con 0x7fa86015b260 2023-11-04T05:29:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 >> v1:172.21.15.3:6789/0 conn(0x7fa86015b260 legacy=0x7fa8600caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c003a00 0x7fa84c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 0x7fa84c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84c004d90 0x7fa84c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600cdee0 con 0x7fa84c004d90 2023-11-04T05:29:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86015baa0 con 0x7fa84c003a00 2023-11-04T05:29:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c00ba10 con 0x7fa8600cee80 2023-11-04T05:29:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa865ac5700 1 --2- 172.21.15.3:0/3594992658 >> v2:172.21.15.188:3300/0 conn(0x7fa8600ceaa0 0x7fa8600cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa8662c6700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84c004d90 0x7fa84c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.944+0000 7fa8652c4700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 0x7fa84c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa865ac5700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c003a00 0x7fa84c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 -- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 msgr2=0x7fa84c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600cee80 0x7fa84c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 -- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c003a00 msgr2=0x7fa84c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c003a00 0x7fa84c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 -- 172.21.15.3:0/3594992658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0035b0 con 0x7fa84c004d90 2023-11-04T05:29:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84c004d90 0x7fa84c005210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa85800a860 tx=0x7fa84c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a288d10a7e878072 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858012600 con 0x7fa84c004d90 2023-11-04T05:29:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858012760 con 0x7fa84c004d90 2023-11-04T05:29:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa864ac3700 1 -- 172.21.15.3:0/3594992658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858012b10 con 0x7fa84c004d90 2023-11-04T05:29:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa867d29700 1 -- 172.21.15.3:0/3594992658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84c004d90 msgr2=0x7fa84c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa867d29700 1 --2- 172.21.15.3:0/3594992658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa84c004d90 0x7fa84c005210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa85800a860 tx=0x7fa84c005880 comp rx=0 tx=0).stop 2023-11-04T05:29:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa8662c6700 1 -- 172.21.15.3:0/3594992658 reap_dead start 2023-11-04T05:29:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa867d29700 1 -- 172.21.15.3:0/3594992658 shutdown_connections 2023-11-04T05:29:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.945+0000 7fa867d29700 1 -- 172.21.15.3:0/3594992658 >> 172.21.15.3:0/3594992658 conn(0x7fa860072560 msgr2=0x7fa860099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.946+0000 7fa867d29700 1 -- 172.21.15.3:0/3594992658 shutdown_connections 2023-11-04T05:29:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.946+0000 7fa867d29700 1 -- 172.21.15.3:0/3594992658 wait complete. 2023-11-04T05:29:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.947+0000 7fa867d29700 1 Processor -- start 2023-11-04T05:29:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.947+0000 7fa867d29700 1 -- start start 2023-11-04T05:29:14.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.947+0000 7fa867d29700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c004d90 0x7fa86014d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 0x7fa8600c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600c7c60 0x7fa860159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c89a0 con 0x7fa8600c6f60 2023-11-04T05:29:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c8ce0 con 0x7fa84c004d90 2023-11-04T05:29:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c8100 con 0x7fa8600c7c60 2023-11-04T05:29:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 0x7fa8600c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 0x7fa8600c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53078/0 (socket says 172.21.15.3:53078) 2023-11-04T05:29:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 -- 172.21.15.3:0/2556194042 learned_addr learned my addr 172.21.15.3:0/2556194042 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 -- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600c7c60 msgr2=0x7fa860159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600c7c60 0x7fa860159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 -- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c004d90 msgr2=0x7fa86014d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c004d90 0x7fa86014d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 -- 172.21.15.3:0/2556194042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600c83e0 con 0x7fa8600c6f60 2023-11-04T05:29:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8652c4700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 0x7fa8600c7720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa854011a20 tx=0x7fa854011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400f940 con 0x7fa8600c6f60 2023-11-04T05:29:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85400faa0 con 0x7fa8600c6f60 2023-11-04T05:29:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8600c86c0 con 0x7fa8600c6f60 2023-11-04T05:29:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400fe50 con 0x7fa8600c6f60 2023-11-04T05:29:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.948+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa860159a80 con 0x7fa8600c6f60 2023-11-04T05:29:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.949+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8600c9010 con 0x7fa8600c6f60 2023-11-04T05:29:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.950+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa854014440 con 0x7fa8600c6f60 2023-11-04T05:29:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.950+0000 7fa8527fc700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa84406af00 0x7fa84406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.950+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa85408a050 con 0x7fa8600c6f60 2023-11-04T05:29:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.950+0000 7fa865ac5700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa84406af00 0x7fa84406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.951+0000 7fa865ac5700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa84406af00 0x7fa84406d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009ba0 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:14.954+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa854054e90 con 0x7fa8600c6f60 2023-11-04T05:29:15.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.154+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8600ccbc0 con 0x7fa8600c6f60 2023-11-04T05:29:15.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.155+0000 7fa8527fc700 1 -- 172.21.15.3:0/2556194042 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa8540584b0 con 0x7fa8600c6f60 2023-11-04T05:29:15.156 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa84406af00 msgr2=0x7fa84406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:15.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa84406af00 0x7fa84406d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009ba0 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2023-11-04T05:29:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 msgr2=0x7fa8600c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 0x7fa8600c7720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa854011a20 tx=0x7fa854011de0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 shutdown_connections 2023-11-04T05:29:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa84406af00 0x7fa84406d3c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84c004d90 0x7fa86014d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8600c7c60 0x7fa860159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 --2- 172.21.15.3:0/2556194042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8600c6f60 0x7fa8600c7720 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 >> 172.21.15.3:0/2556194042 conn(0x7fa860072560 msgr2=0x7fa86015d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 shutdown_connections 2023-11-04T05:29:15.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.157+0000 7fa867d29700 1 -- 172.21.15.3:0/2556194042 wait complete. 2023-11-04T05:29:15.170 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-11-04T05:29:15.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.295+0000 7f5c313ac700 1 Processor -- start 2023-11-04T05:29:15.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.295+0000 7f5c313ac700 1 -- start start 2023-11-04T05:29:15.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c313ac700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c2c1612c0 0x7f5c2c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c313ac700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c0b2e00 0x7f5c2c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c313ac700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0d1f80 0x7f5c2c0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c313ac700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d2950 con 0x7f5c2c0b2e00 2023-11-04T05:29:15.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c313ac700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2c1616a0 con 0x7f5c2c1612c0 2023-11-04T05:29:15.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c313ac700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b2bb0 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c0b2e00 0x7f5c2c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c0b2e00 0x7f5c2c0d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53084/0 (socket says 172.21.15.3:53084) 2023-11-04T05:29:15.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/102892775 learned_addr learned my addr 172.21.15.3:0/102892775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2affd700 1 --2- 172.21.15.3:0/102892775 >> v2:172.21.15.188:3300/0 conn(0x7f5c2c1612c0 0x7f5c2c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/102892775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0d1f80 msgr2=0x7f5c2c0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2b7fe700 1 --2- 172.21.15.3:0/102892775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0d1f80 0x7f5c2c0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 --2- 172.21.15.3:0/102892775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0d1f80 0x7f5c2c0d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/102892775 >> v2:172.21.15.188:3300/0 conn(0x7f5c2c1612c0 msgr2=0x7f5c2c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 --2- 172.21.15.3:0/102892775 >> v2:172.21.15.188:3300/0 conn(0x7f5c2c1612c0 0x7f5c2c06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.296+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/102892775 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0d42c0 con 0x7f5c2c0b2e00 2023-11-04T05:29:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c2affd700 1 --2- 172.21.15.3:0/102892775 >> v2:172.21.15.188:3300/0 conn(0x7f5c2c1612c0 0x7f5c2c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c2a7fc700 1 --2- 172.21.15.3:0/102892775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0b2e00 0x7f5c2c0d1780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c08e0e0 tx=0x7f5c1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c45ad2675ff27 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:15.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c29ffb700 1 -- 172.21.15.3:0/102892775 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00b870 con 0x7f5c2c0b2e00 2023-11-04T05:29:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c29ffb700 1 -- 172.21.15.3:0/102892775 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c00b9d0 con 0x7f5c2c0b2e00 2023-11-04T05:29:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c29ffb700 1 -- 172.21.15.3:0/102892775 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00bd60 con 0x7f5c2c0b2e00 2023-11-04T05:29:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 -- 172.21.15.3:0/102892775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0b2e00 msgr2=0x7f5c2c0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 --2- 172.21.15.3:0/102892775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0b2e00 0x7f5c2c0d1780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c08e0e0 tx=0x7f5c1c009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 -- 172.21.15.3:0/102892775 shutdown_connections 2023-11-04T05:29:15.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 --2- 172.21.15.3:0/102892775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0d1f80 0x7f5c2c0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 --2- 172.21.15.3:0/102892775 >> v2:172.21.15.188:3300/0 conn(0x7f5c2c1612c0 0x7f5c2c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 --2- 172.21.15.3:0/102892775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0b2e00 0x7f5c2c0d1780 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 -- 172.21.15.3:0/102892775 >> 172.21.15.3:0/102892775 conn(0x7f5c2c08aff0 msgr2=0x7f5c2c0d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 -- 172.21.15.3:0/102892775 shutdown_connections 2023-11-04T05:29:15.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.297+0000 7f5c313ac700 1 -- 172.21.15.3:0/102892775 wait complete. 2023-11-04T05:29:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.299+0000 7f5c313ac700 1 Processor -- start 2023-11-04T05:29:15.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.299+0000 7f5c313ac700 1 -- start start 2023-11-04T05:29:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c313ac700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0b2e00 0x7f5c2c0b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c313ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 0x7f5c2c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c313ac700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c2c1612c0 0x7f5c2c14e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c313ac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b3400 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c313ac700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b9390 con 0x7f5c2c1612c0 2023-11-04T05:29:15.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c313ac700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b3650 con 0x7f5c2c0b2e00 2023-11-04T05:29:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c2a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 0x7f5c2c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c2a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 0x7f5c2c0b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53092/0 (socket says 172.21.15.3:53092) 2023-11-04T05:29:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/3398493535 learned_addr learned my addr 172.21.15.3:0/3398493535 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c2b7fe700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c2c1612c0 0x7f5c2c14e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.300+0000 7f5c2affd700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0b2e00 0x7f5c2c0b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0b2e00 msgr2=0x7f5c2c0b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2a7fc700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0b2e00 0x7f5c2c0b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c2c1612c0 msgr2=0x7f5c2c14e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2a7fc700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c2c1612c0 0x7f5c2c14e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:15.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2a7fc700 1 -- 172.21.15.3:0/3398493535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c009550 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2b7fe700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c2c1612c0 0x7f5c2c14e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c2a7fc700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 0x7f5c2c0b4390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c014040 tx=0x7f5c1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c004520 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c01e070 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c0b3930 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0038e0 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.301+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c1f4c90 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.302+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c0b3b40 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.302+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c1c003a40 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.303+0000 7f5c137fe700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c1806af00 0x7f5c1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.303+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5c1c020070 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.303+0000 7f5c2affd700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c1806af00 0x7f5c1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:15.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.304+0000 7f5c2affd700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c1806af00 0x7f5c1806d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000a970 tx=0x7f5c20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:15.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.307+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5c1c055d60 con 0x7f5c2c0d1f80 2023-11-04T05:29:15.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:15.592+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f5c2c074df0 con 0x7f5c2c0d1f80 2023-11-04T05:29:16.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.021+0000 7f5c137fe700 1 -- 172.21.15.3:0/3398493535 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v70) v1 ==== 147+0+0 (secure 0 0 0) 0x7f5c1c02a020 con 0x7f5c2c0d1f80 2023-11-04T05:29:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c1806af00 msgr2=0x7f5c1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c1806af00 0x7f5c1806d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000a970 tx=0x7f5c20008040 comp rx=0 tx=0).stop 2023-11-04T05:29:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 msgr2=0x7f5c2c0b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 0x7f5c2c0b4390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c014040 tx=0x7f5c1c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 shutdown_connections 2023-11-04T05:29:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c1806af00 0x7f5c1806d3c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c2c1612c0 0x7f5c2c14e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2c0b2e00 0x7f5c2c0b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 --2- 172.21.15.3:0/3398493535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0d1f80 0x7f5c2c0b4390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 >> 172.21.15.3:0/3398493535 conn(0x7f5c2c08aff0 msgr2=0x7f5c2c06bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 shutdown_connections 2023-11-04T05:29:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.023+0000 7f5c313ac700 1 -- 172.21.15.3:0/3398493535 wait complete. 2023-11-04T05:29:16.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-04T05:29:16.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.171+0000 7f2ef9691700 1 Processor -- start 2023-11-04T05:29:16.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.171+0000 7f2ef9691700 1 -- start start 2023-11-04T05:29:16.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef9691700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ef415db50 0x7f2ef40cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef9691700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2ef40d2800 0x7f2ef40cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef9691700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef40d1510 con 0x7f2ef40d0330 2023-11-04T05:29:16.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef9691700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40d1760 con 0x7f2ef415db50 2023-11-04T05:29:16.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef9691700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40d1a40 con 0x7f2ef40d2800 2023-11-04T05:29:16.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ef415db50 0x7f2ef40cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ef415db50 0x7f2ef40cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53098/0 (socket says 172.21.15.3:53098) 2023-11-04T05:29:16.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 -- 172.21.15.3:0/1016732132 learned_addr learned my addr 172.21.15.3:0/1016732132 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:16.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 -- 172.21.15.3:0/1016732132 >> v2:172.21.15.188:3300/0 conn(0x7f2ef40d2800 msgr2=0x7f2ef40cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:16.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef27fc700 1 --2- 172.21.15.3:0/1016732132 >> v2:172.21.15.188:3300/0 conn(0x7f2ef40d2800 0x7f2ef40cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 --2- 172.21.15.3:0/1016732132 >> v2:172.21.15.188:3300/0 conn(0x7f2ef40d2800 0x7f2ef40cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 -- 172.21.15.3:0/1016732132 >> v1:172.21.15.188:6789/0 conn(0x7f2ef40d0330 legacy=0x7f2ef40d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:16.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef2ffd700 1 -- 172.21.15.3:0/1016732132 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40d1d20 con 0x7f2ef415db50 2023-11-04T05:29:16.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.172+0000 7f2ef27fc700 1 --2- 172.21.15.3:0/1016732132 >> v2:172.21.15.188:3300/0 conn(0x7f2ef40d2800 0x7f2ef40cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:16.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef2ffd700 1 --2- 172.21.15.3:0/1016732132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef415db50 0x7f2ef40cf080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4161710 tx=0x7f2ee4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfa19cbbb81fa11 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:16.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef1ffb700 1 -- 172.21.15.3:0/1016732132 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4004230 con 0x7f2ef415db50 2023-11-04T05:29:16.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef1ffb700 1 -- 172.21.15.3:0/1016732132 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee4004390 con 0x7f2ef415db50 2023-11-04T05:29:16.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef1ffb700 1 -- 172.21.15.3:0/1016732132 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40036a0 con 0x7f2ef415db50 2023-11-04T05:29:16.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 -- 172.21.15.3:0/1016732132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef415db50 msgr2=0x7f2ef40cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 --2- 172.21.15.3:0/1016732132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef415db50 0x7f2ef40cf080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4161710 tx=0x7f2ee4009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 -- 172.21.15.3:0/1016732132 shutdown_connections 2023-11-04T05:29:16.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 --2- 172.21.15.3:0/1016732132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef415db50 0x7f2ef40cf080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 --2- 172.21.15.3:0/1016732132 >> v2:172.21.15.188:3300/0 conn(0x7f2ef40d2800 0x7f2ef40cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 -- 172.21.15.3:0/1016732132 >> 172.21.15.3:0/1016732132 conn(0x7f2ef408aff0 msgr2=0x7f2ef40d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:16.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 -- 172.21.15.3:0/1016732132 shutdown_connections 2023-11-04T05:29:16.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.173+0000 7f2ef9691700 1 -- 172.21.15.3:0/1016732132 wait complete. 2023-11-04T05:29:16.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.175+0000 7f2ef9691700 1 Processor -- start 2023-11-04T05:29:16.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.175+0000 7f2ef9691700 1 -- start start 2023-11-04T05:29:16.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef9691700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef40d0330 0x7f2ef40b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef9691700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 0x7f2ef40c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef9691700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef415db50 0x7f2ef414ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef9691700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4158fc0 con 0x7f2ef40d2800 2023-11-04T05:29:16.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef9691700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c8b60 con 0x7f2ef415db50 2023-11-04T05:29:16.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef9691700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c8e40 con 0x7f2ef40d0330 2023-11-04T05:29:16.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 0x7f2ef40c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 0x7f2ef40c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53106/0 (socket says 172.21.15.3:53106) 2023-11-04T05:29:16.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef27fc700 1 -- 172.21.15.3:0/779609288 learned_addr learned my addr 172.21.15.3:0/779609288 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:16.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef2ffd700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef40d0330 0x7f2ef40b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.176+0000 7f2ef37fe700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef415db50 0x7f2ef414ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef27fc700 1 -- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef40d0330 msgr2=0x7f2ef40b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef27fc700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef40d0330 0x7f2ef40b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef27fc700 1 -- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef415db50 msgr2=0x7f2ef414ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef27fc700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef415db50 0x7f2ef414ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef27fc700 1 -- 172.21.15.3:0/779609288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4009550 con 0x7f2ef40d2800 2023-11-04T05:29:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef27fc700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 0x7f2ef40c8460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800c750 tx=0x7f2ee800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8011800 con 0x7f2ef40d2800 2023-11-04T05:29:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee8011960 con 0x7f2ef40d2800 2023-11-04T05:29:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8011cd0 con 0x7f2ef40d2800 2023-11-04T05:29:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef40c9120 con 0x7f2ef40d2800 2023-11-04T05:29:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.177+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef40c96e0 con 0x7f2ef40d2800 2023-11-04T05:29:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.178+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef40d1030 con 0x7f2ef40d2800 2023-11-04T05:29:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.178+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ee8021350 con 0x7f2ef40d2800 2023-11-04T05:29:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.179+0000 7f2edb7fe700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2ee006af00 0x7f2ee006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.179+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2ee8089980 con 0x7f2ef40d2800 2023-11-04T05:29:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.179+0000 7f2ef2ffd700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2ee006af00 0x7f2ee006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.180+0000 7f2ef2ffd700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2ee006af00 0x7f2ee006d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009130 tx=0x7f2ee4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:16.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.183+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2ee80545d0 con 0x7f2ef40d2800 2023-11-04T05:29:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.384+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2ef406cb90 con 0x7f2ef40d2800 2023-11-04T05:29:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.389+0000 7f2edb7fe700 1 -- 172.21.15.3:0/779609288 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2ee8057bf0 con 0x7f2ef40d2800 2023-11-04T05:29:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2ee006af00 msgr2=0x7f2ee006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2ee006af00 0x7f2ee006d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009130 tx=0x7f2ee4008030 comp rx=0 tx=0).stop 2023-11-04T05:29:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 msgr2=0x7f2ef40c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 0x7f2ef40c8460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800c750 tx=0x7f2ee800cb10 comp rx=0 tx=0).stop 2023-11-04T05:29:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 shutdown_connections 2023-11-04T05:29:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2ee006af00 0x7f2ee006d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef415db50 0x7f2ef414ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef40d0330 0x7f2ef40b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 --2- 172.21.15.3:0/779609288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ef40d2800 0x7f2ef40c8460 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 >> 172.21.15.3:0/779609288 conn(0x7f2ef408aff0 msgr2=0x7f2ef40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 shutdown_connections 2023-11-04T05:29:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.392+0000 7f2ef9691700 1 -- 172.21.15.3:0/779609288 wait complete. 2023-11-04T05:29:16.405 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-04T05:29:16.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-04T05:29:16.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-04T05:29:16.503 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-04T05:29:16.503 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:16.504 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:16.522 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:16.522 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:16.580 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:16.581 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:16.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 Processor -- start 2023-11-04T05:29:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 -- start start 2023-11-04T05:29:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd32006ab20 0x7fd3200728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3200cc7a0 0x7fd3200cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd320072df0 con 0x7fd32006af00 2023-11-04T05:29:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3200cf050 con 0x7fd32006ab20 2023-11-04T05:29:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.803+0000 7fd325fde700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200cd780 con 0x7fd3200cc7a0 2023-11-04T05:29:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd32006ab20 0x7fd3200728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd32006af00 0x7fd32015ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58462/0 (socket says 172.21.15.3:58462) 2023-11-04T05:29:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd32006ab20 0x7fd3200728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53112/0 (socket says 172.21.15.3:53112) 2023-11-04T05:29:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31ffff700 1 -- 172.21.15.3:0/2428291695 learned_addr learned my addr 172.21.15.3:0/2428291695 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31effd700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3200cc7a0 0x7fd3200cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 -- 172.21.15.3:0/2428291695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3200cc7a0 msgr2=0x7fd3200cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3200cc7a0 0x7fd3200cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 -- 172.21.15.3:0/2428291695 >> v1:172.21.15.3:6789/0 conn(0x7fd32006af00 legacy=0x7fd32015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31e7fc700 1 -- 172.21.15.3:0/2428291695 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492342258 0 0) 0x7fd320072df0 con 0x7fd32006af00 2023-11-04T05:29:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 -- 172.21.15.3:0/2428291695 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3200cda00 con 0x7fd32006ab20 2023-11-04T05:29:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31effd700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3200cc7a0 0x7fd3200cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31f7fe700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd32006ab20 0x7fd3200728b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd310009840 tx=0x7fd310009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be705bc28b72a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31e7fc700 1 -- 172.21.15.3:0/2428291695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310011830 con 0x7fd32006ab20 2023-11-04T05:29:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31e7fc700 1 -- 172.21.15.3:0/2428291695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd310011990 con 0x7fd32006ab20 2023-11-04T05:29:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.804+0000 7fd31e7fc700 1 -- 172.21.15.3:0/2428291695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310011d20 con 0x7fd32006ab20 2023-11-04T05:29:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 -- 172.21.15.3:0/2428291695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd32006ab20 msgr2=0x7fd3200728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd32006ab20 0x7fd3200728b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd310009840 tx=0x7fd310009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 -- 172.21.15.3:0/2428291695 shutdown_connections 2023-11-04T05:29:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd32006ab20 0x7fd3200728b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 --2- 172.21.15.3:0/2428291695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3200cc7a0 0x7fd3200cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 -- 172.21.15.3:0/2428291695 >> 172.21.15.3:0/2428291695 conn(0x7fd32006df90 msgr2=0x7fd32006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 -- 172.21.15.3:0/2428291695 shutdown_connections 2023-11-04T05:29:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.805+0000 7fd325fde700 1 -- 172.21.15.3:0/2428291695 wait complete. 2023-11-04T05:29:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.806+0000 7fd325fde700 1 Processor -- start 2023-11-04T05:29:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.806+0000 7fd325fde700 1 -- start start 2023-11-04T05:29:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd325fde700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32006ab20 0x7fd320154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd325fde700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32006af00 0x7fd32015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd325fde700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 0x7fd320155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd325fde700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32015bca0 con 0x7fd3200cc7a0 2023-11-04T05:29:16.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd325fde700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32015bf80 con 0x7fd32006ab20 2023-11-04T05:29:16.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd325fde700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32015c260 con 0x7fd32006af00 2023-11-04T05:29:16.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 0x7fd320155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 0x7fd320155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53122/0 (socket says 172.21.15.3:53122) 2023-11-04T05:29:16.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd31ffff700 1 -- 172.21.15.3:0/2780478296 learned_addr learned my addr 172.21.15.3:0/2780478296 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd31f7fe700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32006ab20 0x7fd320154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.807+0000 7fd31effd700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32006af00 0x7fd32015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31ffff700 1 -- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32006af00 msgr2=0x7fd32015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31ffff700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32006af00 0x7fd32015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31ffff700 1 -- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32006ab20 msgr2=0x7fd320154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31ffff700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32006ab20 0x7fd320154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31ffff700 1 -- 172.21.15.3:0/2780478296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100094a0 con 0x7fd3200cc7a0 2023-11-04T05:29:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31f7fe700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32006ab20 0x7fd320154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31effd700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32006af00 0x7fd32015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd31ffff700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 0x7fd320155450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd318002230 tx=0x7fd31800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800ac20 con 0x7fd3200cc7a0 2023-11-04T05:29:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31800ada0 con 0x7fd3200cc7a0 2023-11-04T05:29:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32015c540 con 0x7fd3200cc7a0 2023-11-04T05:29:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318007310 con 0x7fd3200cc7a0 2023-11-04T05:29:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.808+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd320154f40 con 0x7fd3200cc7a0 2023-11-04T05:29:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.809+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32015fa50 con 0x7fd3200cc7a0 2023-11-04T05:29:16.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.809+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd318007470 con 0x7fd3200cc7a0 2023-11-04T05:29:16.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.810+0000 7fd307fff700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd30c06af00 0x7fd30c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:16.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.810+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd3180898f0 con 0x7fd3200cc7a0 2023-11-04T05:29:16.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.810+0000 7fd31f7fe700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd30c06af00 0x7fd30c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:16.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.811+0000 7fd31f7fe700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd30c06af00 0x7fd30c06d3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3100090b0 tx=0x7fd31001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:16.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:16.815+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd3180545e0 con 0x7fd3200cc7a0 2023-11-04T05:29:17.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.018+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd320074df0 con 0x7fd3200cc7a0 2023-11-04T05:29:17.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.019+0000 7fd307fff700 1 -- 172.21.15.3:0/2780478296 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7fd318018090 con 0x7fd3200cc7a0 2023-11-04T05:29:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:17.022 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:17.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd30c06af00 msgr2=0x7fd30c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd30c06af00 0x7fd30c06d3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3100090b0 tx=0x7fd31001c040 comp rx=0 tx=0).stop 2023-11-04T05:29:17.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 msgr2=0x7fd320155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 0x7fd320155450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd318002230 tx=0x7fd31800a060 comp rx=0 tx=0).stop 2023-11-04T05:29:17.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 shutdown_connections 2023-11-04T05:29:17.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd30c06af00 0x7fd30c06d3c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32006ab20 0x7fd320154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200cc7a0 0x7fd320155450 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 --2- 172.21.15.3:0/2780478296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32006af00 0x7fd32015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 >> 172.21.15.3:0/2780478296 conn(0x7fd32006df90 msgr2=0x7fd32015f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:17.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 shutdown_connections 2023-11-04T05:29:17.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.022+0000 7fd325fde700 1 -- 172.21.15.3:0/2780478296 wait complete. 2023-11-04T05:29:17.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:17.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.164+0000 7fd4dc974700 1 Processor -- start 2023-11-04T05:29:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.164+0000 7fd4dc974700 1 -- start start 2023-11-04T05:29:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4dc974700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4d81611d0 0x7fd4d80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4dc974700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4d806b230 0x7fd4d80d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4dc974700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d80caca0 con 0x7fd4d80cd4b0 2023-11-04T05:29:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4dc974700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d80ce0a0 con 0x7fd4d806b230 2023-11-04T05:29:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4dc974700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d80cfe40 con 0x7fd4d81611d0 2023-11-04T05:29:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4d806b230 0x7fd4d80d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4d806b230 0x7fd4d80d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53130/0 (socket says 172.21.15.3:53130) 2023-11-04T05:29:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d5d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd4d80cd4b0 0x7fd4d806ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58476/0 (socket says 172.21.15.3:58476) 2023-11-04T05:29:17.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 -- 172.21.15.3:0/2716690299 learned_addr learned my addr 172.21.15.3:0/2716690299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d659c700 1 --2- 172.21.15.3:0/2716690299 >> v2:172.21.15.188:3300/0 conn(0x7fd4d81611d0 0x7fd4d80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 -- 172.21.15.3:0/2716690299 >> v2:172.21.15.188:3300/0 conn(0x7fd4d81611d0 msgr2=0x7fd4d80ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 --2- 172.21.15.3:0/2716690299 >> v2:172.21.15.188:3300/0 conn(0x7fd4d81611d0 0x7fd4d80ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 -- 172.21.15.3:0/2716690299 >> v1:172.21.15.3:6789/0 conn(0x7fd4d80cd4b0 legacy=0x7fd4d806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 -- 172.21.15.3:0/2716690299 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80cf850 con 0x7fd4d806b230 2023-11-04T05:29:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d559a700 1 -- 172.21.15.3:0/2716690299 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549318038 0 0) 0x7fd4d80caca0 con 0x7fd4d80cd4b0 2023-11-04T05:29:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d659c700 1 --2- 172.21.15.3:0/2716690299 >> v2:172.21.15.188:3300/0 conn(0x7fd4d81611d0 0x7fd4d80ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.165+0000 7fd4d6d9d700 1 --2- 172.21.15.3:0/2716690299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d80d02d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8072aa0 tx=0x7fd4d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba5b4c34966a906 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4d559a700 1 -- 172.21.15.3:0/2716690299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000e070 con 0x7fd4d806b230 2023-11-04T05:29:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4d559a700 1 -- 172.21.15.3:0/2716690299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d000b9c0 con 0x7fd4d806b230 2023-11-04T05:29:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4d559a700 1 -- 172.21.15.3:0/2716690299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000bd70 con 0x7fd4d806b230 2023-11-04T05:29:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 -- 172.21.15.3:0/2716690299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 msgr2=0x7fd4d80d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2716690299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d80d02d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8072aa0 tx=0x7fd4d000b330 comp rx=0 tx=0).stop 2023-11-04T05:29:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 -- 172.21.15.3:0/2716690299 shutdown_connections 2023-11-04T05:29:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2716690299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d80d02d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2716690299 >> v2:172.21.15.188:3300/0 conn(0x7fd4d81611d0 0x7fd4d80ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 -- 172.21.15.3:0/2716690299 >> 172.21.15.3:0/2716690299 conn(0x7fd4d8072560 msgr2=0x7fd4d8099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:17.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 -- 172.21.15.3:0/2716690299 shutdown_connections 2023-11-04T05:29:17.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.166+0000 7fd4dc974700 1 -- 172.21.15.3:0/2716690299 wait complete. 2023-11-04T05:29:17.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.168+0000 7fd4dc974700 1 Processor -- start 2023-11-04T05:29:17.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.168+0000 7fd4dc974700 1 -- start start 2023-11-04T05:29:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.168+0000 7fd4dc974700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.168+0000 7fd4dc974700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4d80cd4b0 0x7fd4d806f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.168+0000 7fd4dc974700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4d81611d0 0x7fd4d806f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.168+0000 7fd4dc974700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d815bd70 con 0x7fd4d806b230 2023-11-04T05:29:17.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4dc974700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d814f1b0 con 0x7fd4d80cd4b0 2023-11-04T05:29:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4dc974700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d815f580 con 0x7fd4d81611d0 2023-11-04T05:29:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d815e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53132/0 (socket says 172.21.15.3:53132) 2023-11-04T05:29:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 -- 172.21.15.3:0/2938525597 learned_addr learned my addr 172.21.15.3:0/2938525597 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d5d9b700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4d80cd4b0 0x7fd4d806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d6d9d700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4d81611d0 0x7fd4d806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 -- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4d81611d0 msgr2=0x7fd4d806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4d81611d0 0x7fd4d806f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 -- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4d80cd4b0 msgr2=0x7fd4d806f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4d80cd4b0 0x7fd4d806f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 -- 172.21.15.3:0/2938525597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0009b30 con 0x7fd4d806b230 2023-11-04T05:29:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d6d9d700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4d81611d0 0x7fd4d806f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4d659c700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d815e5e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800e810 tx=0x7fd4c800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800c020 con 0x7fd4d806b230 2023-11-04T05:29:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8004250 con 0x7fd4d806b230 2023-11-04T05:29:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d815efc0 con 0x7fd4d806b230 2023-11-04T05:29:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80036a0 con 0x7fd4d806b230 2023-11-04T05:29:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.169+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d814ee40 con 0x7fd4d806b230 2023-11-04T05:29:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.170+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d8074df0 con 0x7fd4d806b230 2023-11-04T05:29:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.171+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4c8003800 con 0x7fd4d806b230 2023-11-04T05:29:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.171+0000 7fd4c6ffd700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd4b806af00 0x7fd4b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.171+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd4c80893f0 con 0x7fd4d806b230 2023-11-04T05:29:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.171+0000 7fd4d5d9b700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd4b806af00 0x7fd4b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.172+0000 7fd4d5d9b700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd4b806af00 0x7fd4b806d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8160060 tx=0x7fd4cc007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.176+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd4c80540e0 con 0x7fd4d806b230 2023-11-04T05:29:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.374+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4d8073df0 con 0x7fd4d806b230 2023-11-04T05:29:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.375+0000 7fd4c6ffd700 1 -- 172.21.15.3:0/2938525597 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4c8057700 con 0x7fd4d806b230 2023-11-04T05:29:17.376 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd4b806af00 msgr2=0x7fd4b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd4b806af00 0x7fd4b806d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8160060 tx=0x7fd4cc007de0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 msgr2=0x7fd4d815e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d815e5e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800e810 tx=0x7fd4c800eb20 comp rx=0 tx=0).stop 2023-11-04T05:29:17.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 shutdown_connections 2023-11-04T05:29:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd4b806af00 0x7fd4b806d3c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4d81611d0 0x7fd4d806f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4d806b230 0x7fd4d815e5e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 --2- 172.21.15.3:0/2938525597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4d80cd4b0 0x7fd4d806f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 >> 172.21.15.3:0/2938525597 conn(0x7fd4d8072560 msgr2=0x7fd4d8153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 shutdown_connections 2023-11-04T05:29:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.377+0000 7fd4dc974700 1 -- 172.21.15.3:0/2938525597 wait complete. 2023-11-04T05:29:17.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:17.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.521+0000 7f20348af700 1 Processor -- start 2023-11-04T05:29:17.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.521+0000 7f20348af700 1 -- start start 2023-11-04T05:29:17.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.521+0000 7f20348af700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f203015d9a0 0x7f20300d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f20348af700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300ceed0 con 0x7f20300d0800 2023-11-04T05:29:17.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f20348af700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300d1520 con 0x7f20300b2620 2023-11-04T05:29:17.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f20348af700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300cf120 con 0x7f203015d9a0 2023-11-04T05:29:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f203015d9a0 0x7f20300d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20300d0800 0x7f20300d19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58492/0 (socket says 172.21.15.3:58492) 2023-11-04T05:29:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202e59c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f203015d9a0 0x7f20300d3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53140/0 (socket says 172.21.15.3:53140) 2023-11-04T05:29:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202ed9d700 1 -- 172.21.15.3:0/2510232964 learned_addr learned my addr 172.21.15.3:0/2510232964 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202e59c700 1 -- 172.21.15.3:0/2510232964 >> v1:172.21.15.188:6789/0 conn(0x7f20300b2620 legacy=0x7f20300d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202e59c700 1 -- 172.21.15.3:0/2510232964 >> v1:172.21.15.3:6789/0 conn(0x7f20300d0800 legacy=0x7f20300d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202e59c700 1 -- 172.21.15.3:0/2510232964 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300cf400 con 0x7f203015d9a0 2023-11-04T05:29:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202d59a700 1 -- 172.21.15.3:0/2510232964 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360166866 0 0) 0x7f20300ceed0 con 0x7f20300d0800 2023-11-04T05:29:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202e59c700 1 --2- 172.21.15.3:0/2510232964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300d3fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2028009840 tx=0x7f2028009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807ce6250de1c509 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202d59a700 1 -- 172.21.15.3:0/2510232964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028004230 con 0x7f203015d9a0 2023-11-04T05:29:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202d59a700 1 -- 172.21.15.3:0/2510232964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2028004390 con 0x7f203015d9a0 2023-11-04T05:29:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.522+0000 7f202d59a700 1 -- 172.21.15.3:0/2510232964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20280036a0 con 0x7f203015d9a0 2023-11-04T05:29:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 -- 172.21.15.3:0/2510232964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 msgr2=0x7f20300d3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 --2- 172.21.15.3:0/2510232964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300d3fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2028009840 tx=0x7f2028009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 -- 172.21.15.3:0/2510232964 shutdown_connections 2023-11-04T05:29:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 --2- 172.21.15.3:0/2510232964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300d3fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 -- 172.21.15.3:0/2510232964 >> 172.21.15.3:0/2510232964 conn(0x7f203008aff0 msgr2=0x7f20300d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 -- 172.21.15.3:0/2510232964 shutdown_connections 2023-11-04T05:29:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.523+0000 7f20348af700 1 -- 172.21.15.3:0/2510232964 wait complete. 2023-11-04T05:29:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.524+0000 7f20348af700 1 Processor -- start 2023-11-04T05:29:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.524+0000 7f20348af700 1 -- start start 2023-11-04T05:29:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f20348af700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20300b2620 0x7f203015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f20348af700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20300d0800 0x7f203015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f20348af700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f20348af700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203015c240 con 0x7f203015d9a0 2023-11-04T05:29:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f20348af700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300c76f0 con 0x7f20300b2620 2023-11-04T05:29:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f20348af700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300c79d0 con 0x7f20300d0800 2023-11-04T05:29:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53150/0 (socket says 172.21.15.3:53150) 2023-11-04T05:29:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 -- 172.21.15.3:0/2177937794 learned_addr learned my addr 172.21.15.3:0/2177937794 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202e59c700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20300b2620 0x7f203015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202dd9b700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20300d0800 0x7f203015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 -- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20300d0800 msgr2=0x7f203015dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20300d0800 0x7f203015dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 -- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20300b2620 msgr2=0x7f203015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.525+0000 7f202ed9d700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20300b2620 0x7f203015bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f202ed9d700 1 -- 172.21.15.3:0/2177937794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280094a0 con 0x7f203015d9a0 2023-11-04T05:29:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f202e59c700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20300b2620 0x7f203015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f202ed9d700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300c8790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f202000c410 tx=0x7f202000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020004c80 con 0x7f203015d9a0 2023-11-04T05:29:17.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020004de0 con 0x7f203015d9a0 2023-11-04T05:29:17.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20300c7cb0 con 0x7f203015d9a0 2023-11-04T05:29:17.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200072f0 con 0x7f203015d9a0 2023-11-04T05:29:17.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.526+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20300c8270 con 0x7f203015d9a0 2023-11-04T05:29:17.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.527+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20300c8490 con 0x7f203015d9a0 2023-11-04T05:29:17.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.527+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2020007450 con 0x7f203015d9a0 2023-11-04T05:29:17.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.528+0000 7f201effd700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f201006af30 0x7f201006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.528+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2020089690 con 0x7f203015d9a0 2023-11-04T05:29:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.528+0000 7f202e59c700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f201006af30 0x7f201006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.529+0000 7f202e59c700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f201006af30 0x7f201006d3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2028000c00 tx=0x7f202801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.532+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f20200542f0 con 0x7f203015d9a0 2023-11-04T05:29:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.738+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20300b18d0 con 0x7f203015d9a0 2023-11-04T05:29:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.738+0000 7f201effd700 1 -- 172.21.15.3:0/2177937794 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2020057910 con 0x7f203015d9a0 2023-11-04T05:29:17.739 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f201006af30 msgr2=0x7f201006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f201006af30 0x7f201006d3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2028000c00 tx=0x7f202801c040 comp rx=0 tx=0).stop 2023-11-04T05:29:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 msgr2=0x7f20300c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300c8790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f202000c410 tx=0x7f202000c7d0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 shutdown_connections 2023-11-04T05:29:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f201006af30 0x7f201006d3f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20300b2620 0x7f203015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f203015d9a0 0x7f20300c8790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 --2- 172.21.15.3:0/2177937794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20300d0800 0x7f203015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 >> 172.21.15.3:0/2177937794 conn(0x7f203008aff0 msgr2=0x7f20300b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 shutdown_connections 2023-11-04T05:29:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.740+0000 7f20348af700 1 -- 172.21.15.3:0/2177937794 wait complete. 2023-11-04T05:29:17.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.881+0000 7f98d498c700 1 Processor -- start 2023-11-04T05:29:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.881+0000 7f98d498c700 1 -- start start 2023-11-04T05:29:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98d498c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98d00cc100 0x7f98d015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98d498c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d01601a0 0x7f98d006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98d498c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00b0f20 con 0x7f98d0155df0 2023-11-04T05:29:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98d498c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d015c560 con 0x7f98d00cc100 2023-11-04T05:29:17.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98d498c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00cc560 con 0x7f98d01601a0 2023-11-04T05:29:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cdd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98d00cc100 0x7f98d015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98ce59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f98d0155df0 0x7f98d015be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58502/0 (socket says 172.21.15.3:58502) 2023-11-04T05:29:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98ce59c700 1 -- 172.21.15.3:0/71326257 learned_addr learned my addr 172.21.15.3:0/71326257 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98ced9d700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d01601a0 0x7f98d006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cdd9b700 1 -- 172.21.15.3:0/71326257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d01601a0 msgr2=0x7f98d006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cdd9b700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d01601a0 0x7f98d006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cdd9b700 1 -- 172.21.15.3:0/71326257 >> v1:172.21.15.3:6789/0 conn(0x7f98d0155df0 legacy=0x7f98d015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cdd9b700 1 -- 172.21.15.3:0/71326257 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d015f4a0 con 0x7f98d00cc100 2023-11-04T05:29:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cd59a700 1 -- 172.21.15.3:0/71326257 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148739770 0 0) 0x7f98c4001110 con 0x7f98d0155df0 2023-11-04T05:29:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98ced9d700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d01601a0 0x7f98d006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cdd9b700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d00cc100 0x7f98d015fc60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009840 tx=0x7f98c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c13bd269781bbdb server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cd59a700 1 -- 172.21.15.3:0/71326257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0011830 con 0x7f98d00cc100 2023-11-04T05:29:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.882+0000 7f98cd59a700 1 -- 172.21.15.3:0/71326257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0011990 con 0x7f98d00cc100 2023-11-04T05:29:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98cd59a700 1 -- 172.21.15.3:0/71326257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0011d20 con 0x7f98d00cc100 2023-11-04T05:29:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 -- 172.21.15.3:0/71326257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d00cc100 msgr2=0x7f98d015fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d00cc100 0x7f98d015fc60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009840 tx=0x7f98c0009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 -- 172.21.15.3:0/71326257 shutdown_connections 2023-11-04T05:29:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d01601a0 0x7f98d006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 --2- 172.21.15.3:0/71326257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d00cc100 0x7f98d015fc60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 -- 172.21.15.3:0/71326257 >> 172.21.15.3:0/71326257 conn(0x7f98d0072560 msgr2=0x7f98d0099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 -- 172.21.15.3:0/71326257 shutdown_connections 2023-11-04T05:29:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.883+0000 7f98d498c700 1 -- 172.21.15.3:0/71326257 wait complete. 2023-11-04T05:29:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.884+0000 7f98d498c700 1 Processor -- start 2023-11-04T05:29:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 -- start start 2023-11-04T05:29:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d00cc100 0x7f98d01f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 0x7f98d00c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d00b31b0 0x7f98d00b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00b3b80 con 0x7f98d0155df0 2023-11-04T05:29:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00b6510 con 0x7f98d00b31b0 2023-11-04T05:29:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98d498c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00b5470 con 0x7f98d00cc100 2023-11-04T05:29:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98cdd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 0x7f98d00c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98cdd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 0x7f98d00c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53160/0 (socket says 172.21.15.3:53160) 2023-11-04T05:29:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98cdd9b700 1 -- 172.21.15.3:0/1290643791 learned_addr learned my addr 172.21.15.3:0/1290643791 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98ce59c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d00cc100 0x7f98d01f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.885+0000 7f98ced9d700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d00b31b0 0x7f98d00b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98cdd9b700 1 -- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d00cc100 msgr2=0x7f98d01f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98cdd9b700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d00cc100 0x7f98d01f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98cdd9b700 1 -- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d00b31b0 msgr2=0x7f98d00b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98cdd9b700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d00b31b0 0x7f98d00b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98cdd9b700 1 -- 172.21.15.3:0/1290643791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00094a0 con 0x7f98d0155df0 2023-11-04T05:29:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98ce59c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d00cc100 0x7f98d01f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98cdd9b700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 0x7f98d00c3110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98c00090b0 tx=0x7f98c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00036a0 con 0x7f98d0155df0 2023-11-04T05:29:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c001f070 con 0x7f98d0155df0 2023-11-04T05:29:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d00b5750 con 0x7f98d0155df0 2023-11-04T05:29:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0003a00 con 0x7f98d0155df0 2023-11-04T05:29:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.886+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d00b5d10 con 0x7f98d0155df0 2023-11-04T05:29:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.887+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d015c2f0 con 0x7f98d0155df0 2023-11-04T05:29:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.888+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98c0003b60 con 0x7f98d0155df0 2023-11-04T05:29:17.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.888+0000 7f98beffd700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f98b006af00 0x7f98b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:17.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.888+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f98c0021070 con 0x7f98d0155df0 2023-11-04T05:29:17.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.888+0000 7f98ce59c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f98b006af00 0x7f98b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:17.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.889+0000 7f98ce59c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f98b006af00 0x7f98b006d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98c4000f60 tx=0x7f98c4007ae0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:17.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:17.893+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f98c0056080 con 0x7f98d0155df0 2023-11-04T05:29:18.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.091+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98d006b960 con 0x7f98d0155df0 2023-11-04T05:29:18.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.092+0000 7f98beffd700 1 -- 172.21.15.3:0/1290643791 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98c00596a0 con 0x7f98d0155df0 2023-11-04T05:29:18.093 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:18.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f98b006af00 msgr2=0x7f98b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f98b006af00 0x7f98b006d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98c4000f60 tx=0x7f98c4007ae0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 msgr2=0x7f98d00c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 0x7f98d00c3110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98c00090b0 tx=0x7f98c0004750 comp rx=0 tx=0).stop 2023-11-04T05:29:18.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 shutdown_connections 2023-11-04T05:29:18.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f98b006af00 0x7f98b006d3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d00b31b0 0x7f98d00b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d00cc100 0x7f98d01f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 --2- 172.21.15.3:0/1290643791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98d0155df0 0x7f98d00c3110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 >> 172.21.15.3:0/1290643791 conn(0x7f98d0072560 msgr2=0x7f98d015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 shutdown_connections 2023-11-04T05:29:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.094+0000 7f98d498c700 1 -- 172.21.15.3:0/1290643791 wait complete. 2023-11-04T05:29:18.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.234+0000 7fde2773a700 1 Processor -- start 2023-11-04T05:29:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.234+0000 7fde2773a700 1 -- start start 2023-11-04T05:29:18.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde2773a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde200cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde2773a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde200cee80 0x7fde2015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde2773a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde200ccc70 con 0x7fde2015b260 2023-11-04T05:29:18.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde2773a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200cdee0 con 0x7fde200cee80 2023-11-04T05:29:18.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde2773a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2015baa0 con 0x7fde200ceaa0 2023-11-04T05:29:18.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde24cd5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde200cee80 0x7fde2015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde25cd7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fde2015b260 0x7fde200caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58518/0 (socket says 172.21.15.3:58518) 2023-11-04T05:29:18.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.235+0000 7fde25cd7700 1 -- 172.21.15.3:0/1619760912 learned_addr learned my addr 172.21.15.3:0/1619760912 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:18.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde254d6700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde200cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde24cd5700 1 -- 172.21.15.3:0/1619760912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 msgr2=0x7fde200cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:18.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde24cd5700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde200cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde24cd5700 1 -- 172.21.15.3:0/1619760912 >> v1:172.21.15.3:6789/0 conn(0x7fde2015b260 legacy=0x7fde200caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde24cd5700 1 -- 172.21.15.3:0/1619760912 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde200ccec0 con 0x7fde200cee80 2023-11-04T05:29:18.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde17fff700 1 -- 172.21.15.3:0/1619760912 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507970586 0 0) 0x7fde200ccc70 con 0x7fde2015b260 2023-11-04T05:29:18.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde254d6700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde200cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:18.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde24cd5700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde2015ad20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009840 tx=0x7fde1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588cc51aa18f2c3c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:18.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde17fff700 1 -- 172.21.15.3:0/1619760912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011830 con 0x7fde200cee80 2023-11-04T05:29:18.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde17fff700 1 -- 172.21.15.3:0/1619760912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c011990 con 0x7fde200cee80 2023-11-04T05:29:18.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.236+0000 7fde17fff700 1 -- 172.21.15.3:0/1619760912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011d20 con 0x7fde200cee80 2023-11-04T05:29:18.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 -- 172.21.15.3:0/1619760912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 msgr2=0x7fde2015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde2015ad20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009840 tx=0x7fde1c009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:18.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 -- 172.21.15.3:0/1619760912 shutdown_connections 2023-11-04T05:29:18.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde200cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 --2- 172.21.15.3:0/1619760912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde2015ad20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 -- 172.21.15.3:0/1619760912 >> 172.21.15.3:0/1619760912 conn(0x7fde20072560 msgr2=0x7fde20099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:18.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 -- 172.21.15.3:0/1619760912 shutdown_connections 2023-11-04T05:29:18.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.237+0000 7fde2773a700 1 -- 172.21.15.3:0/1619760912 wait complete. 2023-11-04T05:29:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.238+0000 7fde2773a700 1 Processor -- start 2023-11-04T05:29:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.238+0000 7fde2773a700 1 -- start start 2023-11-04T05:29:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde2773a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde201f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde2773a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde200b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde2773a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2015b260 0x7fde200b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde2773a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c6e00 con 0x7fde200cee80 2023-11-04T05:29:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde2773a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c5460 con 0x7fde2015b260 2023-11-04T05:29:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde2773a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c48e0 con 0x7fde200ceaa0 2023-11-04T05:29:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde200b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde200b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53174/0 (socket says 172.21.15.3:53174) 2023-11-04T05:29:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 -- 172.21.15.3:0/2603193318 learned_addr learned my addr 172.21.15.3:0/2603193318 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde254d6700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde201f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde25cd7700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2015b260 0x7fde200b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 -- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 msgr2=0x7fde201f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde201f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 -- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2015b260 msgr2=0x7fde200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2015b260 0x7fde200b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 -- 172.21.15.3:0/2603193318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0094a0 con 0x7fde200cee80 2023-11-04T05:29:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde254d6700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde201f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.239+0000 7fde24cd5700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde200b23e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009b90 tx=0x7fde1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.240+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0036a0 con 0x7fde200cee80 2023-11-04T05:29:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.240+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c01f070 con 0x7fde200cee80 2023-11-04T05:29:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.240+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde200c4bc0 con 0x7fde200cee80 2023-11-04T05:29:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.240+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c003a40 con 0x7fde200cee80 2023-11-04T05:29:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.240+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde200c5180 con 0x7fde200cee80 2023-11-04T05:29:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.241+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde200c8800 con 0x7fde200cee80 2023-11-04T05:29:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.241+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde1c003ba0 con 0x7fde200cee80 2023-11-04T05:29:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.242+0000 7fde15ffb700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde0406af00 0x7fde0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.242+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fde1c021070 con 0x7fde200cee80 2023-11-04T05:29:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.242+0000 7fde254d6700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde0406af00 0x7fde0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.243+0000 7fde254d6700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde0406af00 0x7fde0406d3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde200c73d0 tx=0x7fde10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.247+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fde200c8800 con 0x7fde200cee80 2023-11-04T05:29:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.447+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fde200c8800 con 0x7fde200cee80 2023-11-04T05:29:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.447+0000 7fde15ffb700 1 -- 172.21.15.3:0/2603193318 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fde200c8800 con 0x7fde200cee80 2023-11-04T05:29:18.449 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde0406af00 msgr2=0x7fde0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde0406af00 0x7fde0406d3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde200c73d0 tx=0x7fde10008040 comp rx=0 tx=0).stop 2023-11-04T05:29:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 msgr2=0x7fde200b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde200b23e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009b90 tx=0x7fde1c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:18.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 shutdown_connections 2023-11-04T05:29:18.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fde0406af00 0x7fde0406d3c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde2015b260 0x7fde200b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde200ceaa0 0x7fde201f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 --2- 172.21.15.3:0/2603193318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde200cee80 0x7fde200b23e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 >> 172.21.15.3:0/2603193318 conn(0x7fde20072560 msgr2=0x7fde2006f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:18.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 shutdown_connections 2023-11-04T05:29:18.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.449+0000 7fde2773a700 1 -- 172.21.15.3:0/2603193318 wait complete. 2023-11-04T05:29:18.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T05:29:18.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 Processor -- start 2023-11-04T05:29:18.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 -- start start 2023-11-04T05:29:18.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc6180c6850 0x7fc6180cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6180cd4d0 0x7fc61815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180c4830 con 0x7fc6180c6c30 2023-11-04T05:29:18.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180cc740 con 0x7fc6180cd4d0 2023-11-04T05:29:18.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.588+0000 7fc61ea9d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180cc460 con 0x7fc6180c6850 2023-11-04T05:29:18.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61d03a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6180cd4d0 0x7fc61815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc617fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc6180c6c30 0x7fc6180ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58528/0 (socket says 172.21.15.3:58528) 2023-11-04T05:29:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc617fff700 1 -- 172.21.15.3:0/1613492659 learned_addr learned my addr 172.21.15.3:0/1613492659 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61c839700 1 --2- 172.21.15.3:0/1613492659 >> v2:172.21.15.188:3300/0 conn(0x7fc6180c6850 0x7fc6180cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61d03a700 1 -- 172.21.15.3:0/1613492659 >> v2:172.21.15.188:3300/0 conn(0x7fc6180c6850 msgr2=0x7fc6180cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61d03a700 1 --2- 172.21.15.3:0/1613492659 >> v2:172.21.15.188:3300/0 conn(0x7fc6180c6850 0x7fc6180cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61d03a700 1 -- 172.21.15.3:0/1613492659 >> v1:172.21.15.3:6789/0 conn(0x7fc6180c6c30 legacy=0x7fc6180ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61d03a700 1 -- 172.21.15.3:0/1613492659 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180c41e0 con 0x7fc6180cd4d0 2023-11-04T05:29:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc6177fe700 1 -- 172.21.15.3:0/1613492659 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828876210 0 0) 0x7fc6180c4830 con 0x7fc6180c6c30 2023-11-04T05:29:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc61d03a700 1 --2- 172.21.15.3:0/1613492659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180cd4d0 0x7fc61815a1b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc618072100 tx=0x7fc61000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9b5ee3e67bdebe server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:18.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc6177fe700 1 -- 172.21.15.3:0/1613492659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61000e070 con 0x7fc6180cd4d0 2023-11-04T05:29:18.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc6177fe700 1 -- 172.21.15.3:0/1613492659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61000bbd0 con 0x7fc6180cd4d0 2023-11-04T05:29:18.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.589+0000 7fc6177fe700 1 -- 172.21.15.3:0/1613492659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610012430 con 0x7fc6180cd4d0 2023-11-04T05:29:18.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 -- 172.21.15.3:0/1613492659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180cd4d0 msgr2=0x7fc61815a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/1613492659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180cd4d0 0x7fc61815a1b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc618072100 tx=0x7fc61000b330 comp rx=0 tx=0).stop 2023-11-04T05:29:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 -- 172.21.15.3:0/1613492659 shutdown_connections 2023-11-04T05:29:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/1613492659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180cd4d0 0x7fc61815a1b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/1613492659 >> v2:172.21.15.188:3300/0 conn(0x7fc6180c6850 0x7fc6180cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 -- 172.21.15.3:0/1613492659 >> 172.21.15.3:0/1613492659 conn(0x7fc6180d0560 msgr2=0x7fc6180d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 -- 172.21.15.3:0/1613492659 shutdown_connections 2023-11-04T05:29:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.590+0000 7fc61ea9d700 1 -- 172.21.15.3:0/1613492659 wait complete. 2023-11-04T05:29:18.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.591+0000 7fc61ea9d700 1 Processor -- start 2023-11-04T05:29:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.591+0000 7fc61ea9d700 1 -- start start 2023-11-04T05:29:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61ea9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6180c6850 0x7fc618153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61ea9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 0x7fc6180b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61ea9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6180cd4d0 0x7fc61815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61ea9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180b4c20 con 0x7fc6180c6c30 2023-11-04T05:29:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61ea9d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180b4f00 con 0x7fc6180c6850 2023-11-04T05:29:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61ea9d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180b51e0 con 0x7fc6180cd4d0 2023-11-04T05:29:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 0x7fc6180b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 0x7fc6180b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53186/0 (socket says 172.21.15.3:53186) 2023-11-04T05:29:18.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 -- 172.21.15.3:0/4095171123 learned_addr learned my addr 172.21.15.3:0/4095171123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61c839700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6180c6850 0x7fc618153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61d03a700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6180cd4d0 0x7fc61815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 -- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6180cd4d0 msgr2=0x7fc61815ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6180cd4d0 0x7fc61815ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 -- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6180c6850 msgr2=0x7fc618153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6180c6850 0x7fc618153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc617fff700 1 -- 172.21.15.3:0/4095171123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610009a70 con 0x7fc6180c6c30 2023-11-04T05:29:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.592+0000 7fc61c839700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6180c6850 0x7fc618153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:18.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.593+0000 7fc617fff700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 0x7fc6180b41d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc6040011e0 tx=0x7fc60400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.593+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604004c80 con 0x7fc6180c6c30 2023-11-04T05:29:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.593+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc604004e00 con 0x7fc6180c6c30 2023-11-04T05:29:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.593+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6180b54c0 con 0x7fc6180c6c30 2023-11-04T05:29:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.593+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604007310 con 0x7fc6180c6c30 2023-11-04T05:29:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.593+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc618153fe0 con 0x7fc6180c6c30 2023-11-04T05:29:18.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.594+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6180b59e0 con 0x7fc6180c6c30 2023-11-04T05:29:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.594+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc604007470 con 0x7fc6180c6c30 2023-11-04T05:29:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.595+0000 7fc6157fa700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5fc06af00 0x7fc5fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.595+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc6040897e0 con 0x7fc6180c6c30 2023-11-04T05:29:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.595+0000 7fc61c839700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5fc06af00 0x7fc5fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:18.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.595+0000 7fc61c839700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5fc06af00 0x7fc5fc06d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.599+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fc6180b59e0 con 0x7fc6180c6c30 2023-11-04T05:29:18.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.820+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc6180b59e0 con 0x7fc6180c6c30 2023-11-04T05:29:18.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.820+0000 7fc6157fa700 1 -- 172.21.15.3:0/4095171123 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6180b59e0 con 0x7fc6180c6c30 2023-11-04T05:29:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5fc06af00 msgr2=0x7fc5fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5fc06af00 0x7fc5fc06d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c008040 comp rx=0 tx=0).stop 2023-11-04T05:29:18.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 msgr2=0x7fc6180b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:18.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 0x7fc6180b41d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc6040011e0 tx=0x7fc60400cae0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 shutdown_connections 2023-11-04T05:29:18.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fc5fc06af00 0x7fc5fc06d3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6180c6850 0x7fc618153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6180cd4d0 0x7fc61815ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 --2- 172.21.15.3:0/4095171123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6180c6c30 0x7fc6180b41d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:18.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 >> 172.21.15.3:0/4095171123 conn(0x7fc6180d0560 msgr2=0x7fc61806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:18.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 shutdown_connections 2023-11-04T05:29:18.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:18.822+0000 7fc61ea9d700 1 -- 172.21.15.3:0/4095171123 wait complete. 2023-11-04T05:29:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T05:29:18.835 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-04T05:29:18.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:18.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-04T05:29:18.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:29:18.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:18.881 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-04T05:29:18.937 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:18.938 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:18.959 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:18.960 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:19.020 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:19.020 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:19.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.204+0000 7f81e6012700 1 Processor -- start 2023-11-04T05:29:19.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.204+0000 7f81e6012700 1 -- start start 2023-11-04T05:29:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81e6012700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81e00b5f30 0x7f81e00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81e6012700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81e00b38d0 0x7f81e00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81e6012700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e00d0570 con 0x7f81e006ab20 2023-11-04T05:29:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81e6012700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e00cab90 con 0x7f81e00b5f30 2023-11-04T05:29:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81e6012700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e00cae70 con 0x7f81e00b38d0 2023-11-04T05:29:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81e00b5f30 0x7f81e00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81e00b5f30 0x7f81e00b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53196/0 (socket says 172.21.15.3:53196) 2023-11-04T05:29:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 -- 172.21.15.3:0/3979314763 learned_addr learned my addr 172.21.15.3:0/3979314763 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 -- 172.21.15.3:0/3979314763 >> v2:172.21.15.188:3300/0 conn(0x7f81e00b38d0 msgr2=0x7f81e00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 --2- 172.21.15.3:0/3979314763 >> v2:172.21.15.188:3300/0 conn(0x7f81e00b38d0 0x7f81e00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 -- 172.21.15.3:0/3979314763 >> v1:172.21.15.188:6789/0 conn(0x7f81e006ab20 legacy=0x7f81e006af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.205+0000 7f81deffd700 1 -- 172.21.15.3:0/3979314763 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00cb150 con 0x7f81e00b5f30 2023-11-04T05:29:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81deffd700 1 --2- 172.21.15.3:0/3979314763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b5f30 0x7f81e00b3390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f81e00c3f90 tx=0x7f81d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac46f9681e6185d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81de7fc700 1 -- 172.21.15.3:0/3979314763 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0005e30 con 0x7f81e00b5f30 2023-11-04T05:29:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81de7fc700 1 -- 172.21.15.3:0/3979314763 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0006030 con 0x7f81e00b5f30 2023-11-04T05:29:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81de7fc700 1 -- 172.21.15.3:0/3979314763 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00063e0 con 0x7f81e00b5f30 2023-11-04T05:29:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 -- 172.21.15.3:0/3979314763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b5f30 msgr2=0x7f81e00b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 --2- 172.21.15.3:0/3979314763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b5f30 0x7f81e00b3390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f81e00c3f90 tx=0x7f81d0009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 -- 172.21.15.3:0/3979314763 shutdown_connections 2023-11-04T05:29:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 --2- 172.21.15.3:0/3979314763 >> v2:172.21.15.188:3300/0 conn(0x7f81e00b38d0 0x7f81e00d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 --2- 172.21.15.3:0/3979314763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b5f30 0x7f81e00b3390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 -- 172.21.15.3:0/3979314763 >> 172.21.15.3:0/3979314763 conn(0x7f81e006df90 msgr2=0x7f81e006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 -- 172.21.15.3:0/3979314763 shutdown_connections 2023-11-04T05:29:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.206+0000 7f81e6012700 1 -- 172.21.15.3:0/3979314763 wait complete. 2023-11-04T05:29:19.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 Processor -- start 2023-11-04T05:29:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 -- start start 2023-11-04T05:29:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006ab20 0x7f81e00b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 0x7f81e01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b5f30 0x7f81e00c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e01f7550 con 0x7f81e00b38d0 2023-11-04T05:29:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00b2d40 con 0x7f81e006ab20 2023-11-04T05:29:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.208+0000 7f81e6012700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e01f7270 con 0x7f81e00b5f30 2023-11-04T05:29:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 0x7f81e01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 0x7f81e01553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53202/0 (socket says 172.21.15.3:53202) 2023-11-04T05:29:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 -- 172.21.15.3:0/596973519 learned_addr learned my addr 172.21.15.3:0/596973519 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81df7fe700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006ab20 0x7f81e00b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81dffff700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b5f30 0x7f81e00c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 -- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b5f30 msgr2=0x7f81e00c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b5f30 0x7f81e00c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 -- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006ab20 msgr2=0x7f81e00b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006ab20 0x7f81e00b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 -- 172.21.15.3:0/596973519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0009500 con 0x7f81e00b38d0 2023-11-04T05:29:19.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81df7fe700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006ab20 0x7f81e00b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:19.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81deffd700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 0x7f81e01553b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81d0006000 tx=0x7f81d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0004260 con 0x7f81e00b38d0 2023-11-04T05:29:19.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d00043c0 con 0x7f81e00b38d0 2023-11-04T05:29:19.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e00c8800 con 0x7f81e00b38d0 2023-11-04T05:29:19.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00036a0 con 0x7f81e00b38d0 2023-11-04T05:29:19.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.209+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e00c8dc0 con 0x7f81e00b38d0 2023-11-04T05:29:19.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.210+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e0074df0 con 0x7f81e00b38d0 2023-11-04T05:29:19.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.211+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81d0022070 con 0x7f81e00b38d0 2023-11-04T05:29:19.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.211+0000 7f81d7fff700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f81c006af00 0x7f81c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.211+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f81d008b380 con 0x7f81e00b38d0 2023-11-04T05:29:19.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.211+0000 7f81df7fe700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f81c006af00 0x7f81c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.212+0000 7f81df7fe700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f81c006af00 0x7f81c006d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81c80017d0 tx=0x7f81c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.216+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f81d0056100 con 0x7f81e00b38d0 2023-11-04T05:29:19.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.415+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81e00cc4b0 con 0x7f81e00b38d0 2023-11-04T05:29:19.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.416+0000 7f81d7fff700 1 -- 172.21.15.3:0/596973519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7f81d0059720 con 0x7f81e00b38d0 2023-11-04T05:29:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:19.418 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f81c006af00 msgr2=0x7f81c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f81c006af00 0x7f81c006d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81c80017d0 tx=0x7f81c8009040 comp rx=0 tx=0).stop 2023-11-04T05:29:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 msgr2=0x7f81e01553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 0x7f81e01553b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81d0006000 tx=0x7f81d0004750 comp rx=0 tx=0).stop 2023-11-04T05:29:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 shutdown_connections 2023-11-04T05:29:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f81c006af00 0x7f81c006d3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e006ab20 0x7f81e00b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e00b5f30 0x7f81e00c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 --2- 172.21.15.3:0/596973519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e00b38d0 0x7f81e01553b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 >> 172.21.15.3:0/596973519 conn(0x7f81e006df90 msgr2=0x7f81e015adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.418+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 shutdown_connections 2023-11-04T05:29:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.419+0000 7f81e6012700 1 -- 172.21.15.3:0/596973519 wait complete. 2023-11-04T05:29:19.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:19.433 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:19.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 Processor -- start 2023-11-04T05:29:19.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 -- start start 2023-11-04T05:29:19.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f609c15db50 0x7f609c0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c0d2800 0x7f609c0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609c0d1510 con 0x7f609c0d0330 2023-11-04T05:29:19.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c0d1760 con 0x7f609c15db50 2023-11-04T05:29:19.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.560+0000 7f60a15ab700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0d1a40 con 0x7f609c0d2800 2023-11-04T05:29:19.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f609c15db50 0x7f609c0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f609c15db50 0x7f609c0cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53208/0 (socket says 172.21.15.3:53208) 2023-11-04T05:29:19.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 -- 172.21.15.3:0/1885673833 learned_addr learned my addr 172.21.15.3:0/1885673833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:19.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609a7fc700 1 --2- 172.21.15.3:0/1885673833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c0d2800 0x7f609c0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 -- 172.21.15.3:0/1885673833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c0d2800 msgr2=0x7f609c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:19.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 --2- 172.21.15.3:0/1885673833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c0d2800 0x7f609c0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 -- 172.21.15.3:0/1885673833 >> v1:172.21.15.188:6789/0 conn(0x7f609c0d0330 legacy=0x7f609c1617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 -- 172.21.15.3:0/1885673833 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0d1d20 con 0x7f609c15db50 2023-11-04T05:29:19.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f609affd700 1 --2- 172.21.15.3:0/1885673833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c15db50 0x7f609c0cf080 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f608c0098f0 tx=0x7f608c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3162bbb3cc999df server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f6099ffb700 1 -- 172.21.15.3:0/1885673833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011830 con 0x7f609c15db50 2023-11-04T05:29:19.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f6099ffb700 1 -- 172.21.15.3:0/1885673833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c011990 con 0x7f609c15db50 2023-11-04T05:29:19.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.561+0000 7f6099ffb700 1 -- 172.21.15.3:0/1885673833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011d20 con 0x7f609c15db50 2023-11-04T05:29:19.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 -- 172.21.15.3:0/1885673833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c15db50 msgr2=0x7f609c0cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1885673833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c15db50 0x7f609c0cf080 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f608c0098f0 tx=0x7f608c009c00 comp rx=0 tx=0).stop 2023-11-04T05:29:19.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 -- 172.21.15.3:0/1885673833 shutdown_connections 2023-11-04T05:29:19.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1885673833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c15db50 0x7f609c0cf080 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1885673833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c0d2800 0x7f609c0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 -- 172.21.15.3:0/1885673833 >> 172.21.15.3:0/1885673833 conn(0x7f609c08aff0 msgr2=0x7f609c0d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:19.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 -- 172.21.15.3:0/1885673833 shutdown_connections 2023-11-04T05:29:19.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.562+0000 7f60a15ab700 1 -- 172.21.15.3:0/1885673833 wait complete. 2023-11-04T05:29:19.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.563+0000 7f60a15ab700 1 Processor -- start 2023-11-04T05:29:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.563+0000 7f60a15ab700 1 -- start start 2023-11-04T05:29:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f60a15ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609c0d0330 0x7f609c0b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f60a15ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 0x7f609c1f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f60a15ab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c15db50 0x7f609c1f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f60a15ab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1f4830 con 0x7f609c0d2800 2023-11-04T05:29:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f60a15ab700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0b6360 con 0x7f609c0d0330 2023-11-04T05:29:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f60a15ab700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1f7390 con 0x7f609c15db50 2023-11-04T05:29:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 0x7f609c1f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 0x7f609c1f7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53216/0 (socket says 172.21.15.3:53216) 2023-11-04T05:29:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 -- 172.21.15.3:0/1776075787 learned_addr learned my addr 172.21.15.3:0/1776075787 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609affd700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609c0d0330 0x7f609c0b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609b7fe700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c15db50 0x7f609c1f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 -- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c15db50 msgr2=0x7f609c1f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c15db50 0x7f609c1f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 -- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609c0d0330 msgr2=0x7f609c0b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609c0d0330 0x7f609c0b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609a7fc700 1 -- 172.21.15.3:0/1776075787 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c009550 con 0x7f609c0d2800 2023-11-04T05:29:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.564+0000 7f609b7fe700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c15db50 0x7f609c1f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.565+0000 7f609a7fc700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 0x7f609c1f7820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f609000e9b0 tx=0x7f609000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.565+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090009710 con 0x7f609c0d2800 2023-11-04T05:29:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.565+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000c850 con 0x7f609c0d2800 2023-11-04T05:29:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.565+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c0c9810 con 0x7f609c0d2800 2023-11-04T05:29:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.565+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000cbc0 con 0x7f609c0d2800 2023-11-04T05:29:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.565+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c1f4270 con 0x7f609c0d2800 2023-11-04T05:29:19.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.566+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c0ca880 con 0x7f609c0d2800 2023-11-04T05:29:19.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.566+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f609000cd20 con 0x7f609c0d2800 2023-11-04T05:29:19.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.567+0000 7f60837fe700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f608806af00 0x7f608806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.567+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6090089a00 con 0x7f609c0d2800 2023-11-04T05:29:19.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.567+0000 7f609affd700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f608806af00 0x7f608806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.567+0000 7f609affd700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f608806af00 0x7f608806d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f608c014040 tx=0x7f608c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.571+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f6090054780 con 0x7f609c0d2800 2023-11-04T05:29:19.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.771+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f609c0d1030 con 0x7f609c0d2800 2023-11-04T05:29:19.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.771+0000 7f60837fe700 1 -- 172.21.15.3:0/1776075787 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6090057da0 con 0x7f609c0d2800 2023-11-04T05:29:19.773 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f608806af00 msgr2=0x7f608806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f608806af00 0x7f608806d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f608c014040 tx=0x7f608c008030 comp rx=0 tx=0).stop 2023-11-04T05:29:19.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 msgr2=0x7f609c1f7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 0x7f609c1f7820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f609000e9b0 tx=0x7f609000ed70 comp rx=0 tx=0).stop 2023-11-04T05:29:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 shutdown_connections 2023-11-04T05:29:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f608806af00 0x7f608806d3c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609c0d0330 0x7f609c0b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f609c15db50 0x7f609c1f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 --2- 172.21.15.3:0/1776075787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f609c0d2800 0x7f609c1f7820 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.773+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 >> 172.21.15.3:0/1776075787 conn(0x7f609c08aff0 msgr2=0x7f609c0b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.774+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 shutdown_connections 2023-11-04T05:29:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.774+0000 7f60a15ab700 1 -- 172.21.15.3:0/1776075787 wait complete. 2023-11-04T05:29:19.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:19.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.913+0000 7f8b0c984700 1 Processor -- start 2023-11-04T05:29:19.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.913+0000 7f8b0c984700 1 -- start start 2023-11-04T05:29:19.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b0c984700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8b0806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b0c984700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b080caca0 con 0x7f8b081611d0 2023-11-04T05:29:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b0c984700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b080cfe40 con 0x7f8b080d02d0 2023-11-04T05:29:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b0c984700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080ce0a0 con 0x7f8b0806ab20 2023-11-04T05:29:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b0659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8b081611d0 0x7f8b080ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58570/0 (socket says 172.21.15.3:58570) 2023-11-04T05:29:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b0659c700 1 -- 172.21.15.3:0/1529007889 learned_addr learned my addr 172.21.15.3:0/1529007889 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.914+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8b0806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576106936 0 0) 0x7f8b080caca0 con 0x7f8b081611d0 2023-11-04T05:29:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af00035b0 con 0x7f8b081611d0 2023-11-04T05:29:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470293918 0 0) 0x7f8b080cfe40 con 0x7f8b080d02d0 2023-11-04T05:29:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b080caca0 con 0x7f8b080d02d0 2023-11-04T05:29:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2634022410 0 0) 0x7f8af00035b0 con 0x7f8b081611d0 2023-11-04T05:29:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 msgr2=0x7f8b0806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8b0806afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 >> v1:172.21.15.188:6789/0 conn(0x7f8b080d02d0 legacy=0x7f8b080d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080cf850 con 0x7f8b081611d0 2023-11-04T05:29:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f8afc001670 con 0x7f8b081611d0 2023-11-04T05:29:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8b0806afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 >> v1:172.21.15.3:6789/0 conn(0x7f8b081611d0 legacy=0x7f8b080ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8af0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af0004100 0x7f8af0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8af0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080ce0a0 con 0x7f8af0004d90 2023-11-04T05:29:19.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af000bac0 con 0x7f8af0004100 2023-11-04T05:29:19.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af000bc80 con 0x7f8b0806ab20 2023-11-04T05:29:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b06d9d700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8af0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b0659c700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8af0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af0004100 0x7f8af0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b06d9d700 1 -- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 msgr2=0x7f8af0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b06d9d700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b0806ab20 0x7f8af0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b06d9d700 1 -- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af0004100 msgr2=0x7f8af0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b06d9d700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af0004100 0x7f8af0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.915+0000 7f8b06d9d700 1 -- 172.21.15.3:0/1529007889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80094a0 con 0x7f8af0004d90 2023-11-04T05:29:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b06d9d700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8af0005210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c2f0 tx=0x7f8af0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6de25026951b3c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00001f30 con 0x7f8af0004d90 2023-11-04T05:29:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0000ce90 con 0x7f8af0004d90 2023-11-04T05:29:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0559a700 1 -- 172.21.15.3:0/1529007889 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00003c40 con 0x7f8af0004d90 2023-11-04T05:29:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0c984700 1 -- 172.21.15.3:0/1529007889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 msgr2=0x7f8af0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0c984700 1 --2- 172.21.15.3:0/1529007889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8af0005210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c2f0 tx=0x7f8af0005880 comp rx=0 tx=0).stop 2023-11-04T05:29:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b06d9d700 1 -- 172.21.15.3:0/1529007889 reap_dead start 2023-11-04T05:29:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0c984700 1 -- 172.21.15.3:0/1529007889 shutdown_connections 2023-11-04T05:29:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0c984700 1 -- 172.21.15.3:0/1529007889 >> 172.21.15.3:0/1529007889 conn(0x7f8b08072560 msgr2=0x7f8b08099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0c984700 1 -- 172.21.15.3:0/1529007889 shutdown_connections 2023-11-04T05:29:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.916+0000 7f8b0c984700 1 -- 172.21.15.3:0/1529007889 wait complete. 2023-11-04T05:29:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 Processor -- start 2023-11-04T05:29:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 -- start start 2023-11-04T05:29:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8b080ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b08153980 0x7f8b080c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b080ca780 0x7f8b080c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08153690 con 0x7f8af0004d90 2023-11-04T05:29:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08152df0 con 0x7f8b08153980 2023-11-04T05:29:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.918+0000 7f8b0c984700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b081530d0 con 0x7f8b080ca780 2023-11-04T05:29:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8b080ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8b080ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53234/0 (socket says 172.21.15.3:53234) 2023-11-04T05:29:19.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 -- 172.21.15.3:0/2304165490 learned_addr learned my addr 172.21.15.3:0/2304165490 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b08153980 0x7f8b080c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b06d9d700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b080ca780 0x7f8b080c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 -- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b080ca780 msgr2=0x7f8b080c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b080ca780 0x7f8b080c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 -- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b08153980 msgr2=0x7f8b080c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b08153980 0x7f8b080c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 -- 172.21.15.3:0/2304165490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b081533b0 con 0x7f8af0004d90 2023-11-04T05:29:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b06d9d700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b080ca780 0x7f8b080c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:19.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b08153980 0x7f8b080c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0659c700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8b080ca240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8afc007310 tx=0x7f8afc0076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0077f0 con 0x7f8af0004d90 2023-11-04T05:29:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc00f870 con 0x7f8af0004d90 2023-11-04T05:29:19.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00fc40 con 0x7f8af0004d90 2023-11-04T05:29:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0815ee60 con 0x7f8af0004d90 2023-11-04T05:29:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.919+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b080c5f60 con 0x7f8af0004d90 2023-11-04T05:29:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.920+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b080c6180 con 0x7f8af0004d90 2023-11-04T05:29:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.921+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8afc01d460 con 0x7f8af0004d90 2023-11-04T05:29:19.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.921+0000 7f8af6ffd700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ae806af00 0x7f8ae806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:19.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.921+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8afc089990 con 0x7f8af0004d90 2023-11-04T05:29:19.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.921+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ae806af00 0x7f8ae806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:19.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.922+0000 7f8b05d9b700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ae806af00 0x7f8ae806d3c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b08153da0 tx=0x7f8af8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:19.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:19.926+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8afc054710 con 0x7f8af0004d90 2023-11-04T05:29:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.125+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b08073150 con 0x7f8af0004d90 2023-11-04T05:29:20.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.125+0000 7f8af6ffd700 1 -- 172.21.15.3:0/2304165490 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8afc057d30 con 0x7f8af0004d90 2023-11-04T05:29:20.128 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:20.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ae806af00 msgr2=0x7f8ae806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ae806af00 0x7f8ae806d3c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b08153da0 tx=0x7f8af8008040 comp rx=0 tx=0).stop 2023-11-04T05:29:20.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 msgr2=0x7f8b080ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8b080ca240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8afc007310 tx=0x7f8afc0076d0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 shutdown_connections 2023-11-04T05:29:20.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ae806af00 0x7f8ae806d3c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b080ca780 0x7f8b080c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af0004d90 0x7f8b080ca240 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.127+0000 7f8b0c984700 1 --2- 172.21.15.3:0/2304165490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b08153980 0x7f8b080c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.128+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 >> 172.21.15.3:0/2304165490 conn(0x7f8b08072560 msgr2=0x7f8b080cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:20.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.128+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 shutdown_connections 2023-11-04T05:29:20.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.128+0000 7f8b0c984700 1 -- 172.21.15.3:0/2304165490 wait complete. 2023-11-04T05:29:20.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.268+0000 7fadd1565700 1 Processor -- start 2023-11-04T05:29:20.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.268+0000 7fadd1565700 1 -- start start 2023-11-04T05:29:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadd1565700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadcc15db50 0x7fadcc0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadd1565700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fadcc0d2800 0x7fadcc0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadd1565700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0d1510 con 0x7fadcc0d0330 2023-11-04T05:29:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadd1565700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0d1760 con 0x7fadcc15db50 2023-11-04T05:29:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadd1565700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0d1a40 con 0x7fadcc0d2800 2023-11-04T05:29:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadcc15db50 0x7fadcc0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fadcc0d0330 0x7fadcc0d0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58592/0 (socket says 172.21.15.3:58592) 2023-11-04T05:29:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fadcc15db50 0x7fadcc0cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53242/0 (socket says 172.21.15.3:53242) 2023-11-04T05:29:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcb7fe700 1 -- 172.21.15.3:0/1625765632 learned_addr learned my addr 172.21.15.3:0/1625765632 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadca7fc700 1 --2- 172.21.15.3:0/1625765632 >> v2:172.21.15.188:3300/0 conn(0x7fadcc0d2800 0x7fadcc0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 -- 172.21.15.3:0/1625765632 >> v2:172.21.15.188:3300/0 conn(0x7fadcc0d2800 msgr2=0x7fadcc0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 --2- 172.21.15.3:0/1625765632 >> v2:172.21.15.188:3300/0 conn(0x7fadcc0d2800 0x7fadcc0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 -- 172.21.15.3:0/1625765632 >> v1:172.21.15.3:6789/0 conn(0x7fadcc0d0330 legacy=0x7fadcc0d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadc9ffb700 1 -- 172.21.15.3:0/1625765632 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297365879 0 0) 0x7fadcc0d1510 con 0x7fadcc0d0330 2023-11-04T05:29:20.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 -- 172.21.15.3:0/1625765632 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0d1d20 con 0x7fadcc15db50 2023-11-04T05:29:20.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.269+0000 7fadcaffd700 1 --2- 172.21.15.3:0/1625765632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc15db50 0x7fadcc0cf080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fadcc161710 tx=0x7fadc0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f463abb3a6de60 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadc9ffb700 1 -- 172.21.15.3:0/1625765632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011830 con 0x7fadcc15db50 2023-11-04T05:29:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadc9ffb700 1 -- 172.21.15.3:0/1625765632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0011990 con 0x7fadcc15db50 2023-11-04T05:29:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadc9ffb700 1 -- 172.21.15.3:0/1625765632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011d20 con 0x7fadcc15db50 2023-11-04T05:29:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 -- 172.21.15.3:0/1625765632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc15db50 msgr2=0x7fadcc0cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 --2- 172.21.15.3:0/1625765632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc15db50 0x7fadcc0cf080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fadcc161710 tx=0x7fadc0009b20 comp rx=0 tx=0).stop 2023-11-04T05:29:20.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 -- 172.21.15.3:0/1625765632 shutdown_connections 2023-11-04T05:29:20.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 --2- 172.21.15.3:0/1625765632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc15db50 0x7fadcc0cf080 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 --2- 172.21.15.3:0/1625765632 >> v2:172.21.15.188:3300/0 conn(0x7fadcc0d2800 0x7fadcc0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 -- 172.21.15.3:0/1625765632 >> 172.21.15.3:0/1625765632 conn(0x7fadcc08aff0 msgr2=0x7fadcc0d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:20.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 -- 172.21.15.3:0/1625765632 shutdown_connections 2023-11-04T05:29:20.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.270+0000 7fadd1565700 1 -- 172.21.15.3:0/1625765632 wait complete. 2023-11-04T05:29:20.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.271+0000 7fadd1565700 1 Processor -- start 2023-11-04T05:29:20.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 -- start start 2023-11-04T05:29:20.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 0x7fadcc0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadcc0d2800 0x7fadcc1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadcc15db50 0x7fadcc15fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc1f41b0 con 0x7fadcc0d0330 2023-11-04T05:29:20.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc1602a0 con 0x7fadcc0d2800 2023-11-04T05:29:20.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadd1565700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0caab0 con 0x7fadcc15db50 2023-11-04T05:29:20.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadcaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 0x7fadcc0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadcaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 0x7fadcc0cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53248/0 (socket says 172.21.15.3:53248) 2023-11-04T05:29:20.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadcaffd700 1 -- 172.21.15.3:0/3111233029 learned_addr learned my addr 172.21.15.3:0/3111233029 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:20.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.272+0000 7fadca7fc700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadcc0d2800 0x7fadcc1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcb7fe700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadcc15db50 0x7fadcc15fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcaffd700 1 -- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadcc15db50 msgr2=0x7fadcc15fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcaffd700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadcc15db50 0x7fadcc15fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcaffd700 1 -- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadcc0d2800 msgr2=0x7fadcc1f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcaffd700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadcc0d2800 0x7fadcc1f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcaffd700 1 -- 172.21.15.3:0/3111233029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00094a0 con 0x7fadcc0d0330 2023-11-04T05:29:20.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadca7fc700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadcc0d2800 0x7fadcc1f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:20.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadcaffd700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 0x7fadcc0cc4f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fadc0014040 tx=0x7fadc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc00038a0 con 0x7fadcc0d0330 2023-11-04T05:29:20.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc001e070 con 0x7fadcc0d0330 2023-11-04T05:29:20.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0003c20 con 0x7fadcc0d0330 2023-11-04T05:29:20.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc0cad90 con 0x7fadcc0d0330 2023-11-04T05:29:20.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.273+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc0cb350 con 0x7fadcc0d0330 2023-11-04T05:29:20.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.274+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0d1030 con 0x7fadcc0d0330 2023-11-04T05:29:20.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.274+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadc002d450 con 0x7fadcc0d0330 2023-11-04T05:29:20.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.275+0000 7fadb37fe700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fadb806af00 0x7fadb806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.275+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fadc008a990 con 0x7fadcc0d0330 2023-11-04T05:29:20.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.275+0000 7fadca7fc700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fadb806af00 0x7fadb806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.276+0000 7fadca7fc700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fadb806af00 0x7fadb806d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fadcc15b970 tx=0x7fadbc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.279+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fadc0055710 con 0x7fadcc0d0330 2023-11-04T05:29:20.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.480+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadcc0d45e0 con 0x7fadcc0d0330 2023-11-04T05:29:20.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.480+0000 7fadb37fe700 1 -- 172.21.15.3:0/3111233029 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadc0058d30 con 0x7fadcc0d0330 2023-11-04T05:29:20.481 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:20.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fadb806af00 msgr2=0x7fadb806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fadb806af00 0x7fadb806d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fadcc15b970 tx=0x7fadbc008040 comp rx=0 tx=0).stop 2023-11-04T05:29:20.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 msgr2=0x7fadcc0cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 0x7fadcc0cc4f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fadc0014040 tx=0x7fadc0004750 comp rx=0 tx=0).stop 2023-11-04T05:29:20.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 shutdown_connections 2023-11-04T05:29:20.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fadb806af00 0x7fadb806d3c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadcc15db50 0x7fadcc15fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadcc0d0330 0x7fadcc0cc4f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 --2- 172.21.15.3:0/3111233029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadcc0d2800 0x7fadcc1f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 >> 172.21.15.3:0/3111233029 conn(0x7fadcc08aff0 msgr2=0x7fadcc0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:20.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 shutdown_connections 2023-11-04T05:29:20.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.482+0000 7fadd1565700 1 -- 172.21.15.3:0/3111233029 wait complete. 2023-11-04T05:29:20.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:20.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.623+0000 7f2b882c6700 1 Processor -- start 2023-11-04T05:29:20.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.623+0000 7f2b882c6700 1 -- start start 2023-11-04T05:29:20.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b882c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b8006e2f0 0x7f2b800668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b882c6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b800b2f30 0x7f2b80066e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b882c6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b800cdce0 con 0x7f2b8015ddf0 2023-11-04T05:29:20.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b882c6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b800cc590 con 0x7f2b8006e2f0 2023-11-04T05:29:20.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b882c6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8015e1d0 con 0x7f2b800b2f30 2023-11-04T05:29:20.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b85861700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b8006e2f0 0x7f2b800668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b85861700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b8006e2f0 0x7f2b800668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53256/0 (socket says 172.21.15.3:53256) 2023-11-04T05:29:20.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b85861700 1 -- 172.21.15.3:0/3551574507 learned_addr learned my addr 172.21.15.3:0/3551574507 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:20.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.624+0000 7f2b86863700 1 --2- 172.21.15.3:0/3551574507 >> v2:172.21.15.188:3300/0 conn(0x7f2b800b2f30 0x7f2b80066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85861700 1 -- 172.21.15.3:0/3551574507 >> v2:172.21.15.188:3300/0 conn(0x7f2b800b2f30 msgr2=0x7f2b80066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:20.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85861700 1 --2- 172.21.15.3:0/3551574507 >> v2:172.21.15.188:3300/0 conn(0x7f2b800b2f30 0x7f2b80066e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85861700 1 -- 172.21.15.3:0/3551574507 >> v1:172.21.15.188:6789/0 conn(0x7f2b8015ddf0 legacy=0x7f2b800cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85861700 1 -- 172.21.15.3:0/3551574507 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800cdfc0 con 0x7f2b8006e2f0 2023-11-04T05:29:20.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85861700 1 --2- 172.21.15.3:0/3551574507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b8006e2f0 0x7f2b800668e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b8006e770 tx=0x7f2b70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd05dadb42ded2a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85060700 1 -- 172.21.15.3:0/3551574507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70011830 con 0x7f2b8006e2f0 2023-11-04T05:29:20.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85060700 1 -- 172.21.15.3:0/3551574507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70011990 con 0x7f2b8006e2f0 2023-11-04T05:29:20.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.625+0000 7f2b85060700 1 -- 172.21.15.3:0/3551574507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70011d20 con 0x7f2b8006e2f0 2023-11-04T05:29:20.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 -- 172.21.15.3:0/3551574507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b8006e2f0 msgr2=0x7f2b800668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3551574507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b8006e2f0 0x7f2b800668e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b8006e770 tx=0x7f2b70009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 -- 172.21.15.3:0/3551574507 shutdown_connections 2023-11-04T05:29:20.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3551574507 >> v2:172.21.15.188:3300/0 conn(0x7f2b800b2f30 0x7f2b80066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3551574507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b8006e2f0 0x7f2b800668e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 -- 172.21.15.3:0/3551574507 >> 172.21.15.3:0/3551574507 conn(0x7f2b800cc860 msgr2=0x7f2b800ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 -- 172.21.15.3:0/3551574507 shutdown_connections 2023-11-04T05:29:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.626+0000 7f2b882c6700 1 -- 172.21.15.3:0/3551574507 wait complete. 2023-11-04T05:29:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.627+0000 7f2b882c6700 1 Processor -- start 2023-11-04T05:29:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.627+0000 7f2b882c6700 1 -- start start 2023-11-04T05:29:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8006e2f0 0x7f2b80145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 0x7f2b80099f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8015ddf0 0x7f2b801573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80146120 con 0x7f2b800b2f30 2023-11-04T05:29:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801f1bc0 con 0x7f2b8015ddf0 2023-11-04T05:29:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801f1ea0 con 0x7f2b8006e2f0 2023-11-04T05:29:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 0x7f2b80099f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 0x7f2b80099f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53264/0 (socket says 172.21.15.3:53264) 2023-11-04T05:29:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 -- 172.21.15.3:0/3423384059 learned_addr learned my addr 172.21.15.3:0/3423384059 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b86062700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8006e2f0 0x7f2b80145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b86863700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8015ddf0 0x7f2b801573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 -- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8006e2f0 msgr2=0x7f2b80145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8006e2f0 0x7f2b80145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 -- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8015ddf0 msgr2=0x7f2b801573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8015ddf0 0x7f2b801573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 -- 172.21.15.3:0/3423384059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70009550 con 0x7f2b800b2f30 2023-11-04T05:29:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b85861700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 0x7f2b80099f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b70000c00 tx=0x7f2b70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70004480 con 0x7f2b800b2f30 2023-11-04T05:29:20.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7001f070 con 0x7f2b800b2f30 2023-11-04T05:29:20.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b801f2180 con 0x7f2b800b2f30 2023-11-04T05:29:20.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700038c0 con 0x7f2b800b2f30 2023-11-04T05:29:20.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.628+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8009a470 con 0x7f2b800b2f30 2023-11-04T05:29:20.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.629+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b80074df0 con 0x7f2b800b2f30 2023-11-04T05:29:20.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.630+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b70021030 con 0x7f2b800b2f30 2023-11-04T05:29:20.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.630+0000 7f2b76ffd700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b6406af00 0x7f2b6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.630+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b7008ae50 con 0x7f2b800b2f30 2023-11-04T05:29:20.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.631+0000 7f2b86062700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b6406af00 0x7f2b6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.631+0000 7f2b86062700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b6406af00 0x7f2b6406d3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2b80158060 tx=0x7f2b7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.635+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2b70055cf0 con 0x7f2b800b2f30 2023-11-04T05:29:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.837+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2b80099d10 con 0x7f2b800b2f30 2023-11-04T05:29:20.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.837+0000 7f2b76ffd700 1 -- 172.21.15.3:0/3423384059 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2b70059310 con 0x7f2b800b2f30 2023-11-04T05:29:20.839 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b6406af00 msgr2=0x7f2b6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b6406af00 0x7f2b6406d3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2b80158060 tx=0x7f2b7c009040 comp rx=0 tx=0).stop 2023-11-04T05:29:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 msgr2=0x7f2b80099f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 0x7f2b80099f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b70000c00 tx=0x7f2b70004750 comp rx=0 tx=0).stop 2023-11-04T05:29:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 shutdown_connections 2023-11-04T05:29:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f2b6406af00 0x7f2b6406d3c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8015ddf0 0x7f2b801573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8006e2f0 0x7f2b80145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 --2- 172.21.15.3:0/3423384059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b800b2f30 0x7f2b80099f30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 >> 172.21.15.3:0/3423384059 conn(0x7f2b800cc860 msgr2=0x7f2b8015e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 shutdown_connections 2023-11-04T05:29:20.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.839+0000 7f2b882c6700 1 -- 172.21.15.3:0/3423384059 wait complete. 2023-11-04T05:29:20.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T05:29:20.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 Processor -- start 2023-11-04T05:29:20.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 -- start start 2023-11-04T05:29:20.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bbc15bbc0 0x7f8bbc0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8bbc0cec30 0x7f8bbc165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc15c370 con 0x7f8bbc0b58e0 2023-11-04T05:29:20.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc15c650 con 0x7f8bbc15bbc0 2023-11-04T05:29:20.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.978+0000 7f8bc30ae700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc15c930 con 0x7f8bbc0cec30 2023-11-04T05:29:20.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bbc15bbc0 0x7f8bbc0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bbc15bbc0 0x7f8bbc0b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53270/0 (socket says 172.21.15.3:53270) 2023-11-04T05:29:20.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 -- 172.21.15.3:0/3695064941 learned_addr learned my addr 172.21.15.3:0/3695064941 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:20.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bbbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8bbc0b58e0 0x7f8bbc06bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58618/0 (socket says 172.21.15.3:58618) 2023-11-04T05:29:20.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc164b700 1 --2- 172.21.15.3:0/3695064941 >> v2:172.21.15.188:3300/0 conn(0x7f8bbc0cec30 0x7f8bbc165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 -- 172.21.15.3:0/3695064941 >> v2:172.21.15.188:3300/0 conn(0x7f8bbc0cec30 msgr2=0x7f8bbc165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:20.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 --2- 172.21.15.3:0/3695064941 >> v2:172.21.15.188:3300/0 conn(0x7f8bbc0cec30 0x7f8bbc165660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 -- 172.21.15.3:0/3695064941 >> v1:172.21.15.3:6789/0 conn(0x7f8bbc0b58e0 legacy=0x7f8bbc06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bbb7fe700 1 -- 172.21.15.3:0/3695064941 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454324908 0 0) 0x7f8bbc15c370 con 0x7f8bbc0b58e0 2023-11-04T05:29:20.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 -- 172.21.15.3:0/3695064941 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc15cc10 con 0x7f8bbc15bbc0 2023-11-04T05:29:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bc0e4a700 1 --2- 172.21.15.3:0/3695064941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc15bbc0 0x7f8bbc0b53a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0af7d0 tx=0x7f8bb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa2c31d3ce74ba9 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bbb7fe700 1 -- 172.21.15.3:0/3695064941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000b870 con 0x7f8bbc15bbc0 2023-11-04T05:29:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bbb7fe700 1 -- 172.21.15.3:0/3695064941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb000b9d0 con 0x7f8bbc15bbc0 2023-11-04T05:29:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.979+0000 7f8bbb7fe700 1 -- 172.21.15.3:0/3695064941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000bd60 con 0x7f8bbc15bbc0 2023-11-04T05:29:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 -- 172.21.15.3:0/3695064941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc15bbc0 msgr2=0x7f8bbc0b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/3695064941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc15bbc0 0x7f8bbc0b53a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0af7d0 tx=0x7f8bb0009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 -- 172.21.15.3:0/3695064941 shutdown_connections 2023-11-04T05:29:20.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/3695064941 >> v2:172.21.15.188:3300/0 conn(0x7f8bbc0cec30 0x7f8bbc165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/3695064941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc15bbc0 0x7f8bbc0b53a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 -- 172.21.15.3:0/3695064941 >> 172.21.15.3:0/3695064941 conn(0x7f8bbc0716c0 msgr2=0x7f8bbc071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:20.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 -- 172.21.15.3:0/3695064941 shutdown_connections 2023-11-04T05:29:20.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.980+0000 7f8bc30ae700 1 -- 172.21.15.3:0/3695064941 wait complete. 2023-11-04T05:29:20.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.981+0000 7f8bc30ae700 1 Processor -- start 2023-11-04T05:29:20.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.981+0000 7f8bc30ae700 1 -- start start 2023-11-04T05:29:20.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc30ae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bbc0b58e0 0x7f8bbc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc30ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 0x7f8bbc1f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc30ae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bbc15bbc0 0x7f8bbc1f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc30ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0b29a0 con 0x7f8bbc0cec30 2023-11-04T05:29:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc30ae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc06e8d0 con 0x7f8bbc0b58e0 2023-11-04T05:29:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc30ae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc06ebb0 con 0x7f8bbc15bbc0 2023-11-04T05:29:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 0x7f8bbc1f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 0x7f8bbc1f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53276/0 (socket says 172.21.15.3:53276) 2023-11-04T05:29:20.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 -- 172.21.15.3:0/2166940877 learned_addr learned my addr 172.21.15.3:0/2166940877 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc0e4a700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bbc0b58e0 0x7f8bbc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bc164b700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bbc15bbc0 0x7f8bbc1f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 -- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bbc15bbc0 msgr2=0x7f8bbc1f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bbc15bbc0 0x7f8bbc1f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 -- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bbc0b58e0 msgr2=0x7f8bbc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bbc0b58e0 0x7f8bbc06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.982+0000 7f8bbbfff700 1 -- 172.21.15.3:0/2166940877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0009550 con 0x7f8bbc0cec30 2023-11-04T05:29:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.983+0000 7f8bbbfff700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 0x7f8bbc1f3660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c550 tx=0x7f8ba800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.983+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004c80 con 0x7f8bbc0cec30 2023-11-04T05:29:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.983+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8004e00 con 0x7f8bbc0cec30 2023-11-04T05:29:20.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.983+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc0b2bf0 con 0x7f8bbc0cec30 2023-11-04T05:29:20.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.983+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8007310 con 0x7f8bbc0cec30 2023-11-04T05:29:20.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.983+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc1e5110 con 0x7f8bbc0cec30 2023-11-04T05:29:20.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.984+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc0b3110 con 0x7f8bbc0cec30 2023-11-04T05:29:20.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.985+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ba8007470 con 0x7f8bbc0cec30 2023-11-04T05:29:20.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.985+0000 7f8bb97fa700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ba006af00 0x7f8ba006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:20.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.985+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8ba8089570 con 0x7f8bbc0cec30 2023-11-04T05:29:20.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.985+0000 7f8bc0e4a700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ba006af00 0x7f8ba006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:20.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.986+0000 7f8bc0e4a700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ba006af00 0x7f8ba006d3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0000c00 tx=0x7f8bb0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:20.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:20.990+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8bbc0b3110 con 0x7f8bbc0cec30 2023-11-04T05:29:21.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.205+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8bbc0b3110 con 0x7f8bbc0cec30 2023-11-04T05:29:21.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.205+0000 7f8bb97fa700 1 -- 172.21.15.3:0/2166940877 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8bbc0b3110 con 0x7f8bbc0cec30 2023-11-04T05:29:21.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ba006af00 msgr2=0x7f8ba006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ba006af00 0x7f8ba006d3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0000c00 tx=0x7f8bb0008030 comp rx=0 tx=0).stop 2023-11-04T05:29:21.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 msgr2=0x7f8bbc1f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 0x7f8bbc1f3660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c550 tx=0x7f8ba800c860 comp rx=0 tx=0).stop 2023-11-04T05:29:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 shutdown_connections 2023-11-04T05:29:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8ba006af00 0x7f8ba006d3c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bbc0b58e0 0x7f8bbc06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bbc15bbc0 0x7f8bbc1f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 --2- 172.21.15.3:0/2166940877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bbc0cec30 0x7f8bbc1f3660 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 >> 172.21.15.3:0/2166940877 conn(0x7f8bbc0716c0 msgr2=0x7f8bbc157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:21.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 shutdown_connections 2023-11-04T05:29:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.207+0000 7f8bc30ae700 1 -- 172.21.15.3:0/2166940877 wait complete. 2023-11-04T05:29:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T05:29:21.221 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-04T05:29:21.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:21.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-04T05:29:21.270 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:29:21.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:21.271 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-04T05:29:21.301 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:21.301 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:21.319 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:21.320 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:21.378 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:21.379 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:21.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:21.531 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:29:21.533 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:29:21.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.601+0000 7f4d09765700 1 Processor -- start 2023-11-04T05:29:21.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.601+0000 7f4d09765700 1 -- start start 2023-11-04T05:29:21.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d09765700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d040cc7a0 0x7f4d040728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:21.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d09765700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d04072df0 con 0x7f4d0406af00 2023-11-04T05:29:21.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d09765700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040cda00 con 0x7f4d0406ab20 2023-11-04T05:29:21.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d09765700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040cf050 con 0x7f4d040cc7a0 2023-11-04T05:29:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d027fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d040cc7a0 0x7f4d040728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d037fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d0406af00 0x7f4d040d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58634/0 (socket says 172.21.15.3:58634) 2023-11-04T05:29:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.602+0000 7f4d037fe700 1 -- 172.21.15.3:0/1355781615 learned_addr learned my addr 172.21.15.3:0/1355781615 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d027fc700 1 -- 172.21.15.3:0/1355781615 >> v1:172.21.15.188:6789/0 conn(0x7f4d0406ab20 legacy=0x7f4d040cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:21.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d027fc700 1 -- 172.21.15.3:0/1355781615 >> v1:172.21.15.3:6789/0 conn(0x7f4d0406af00 legacy=0x7f4d040d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d01ffb700 1 -- 172.21.15.3:0/1355781615 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875849782 0 0) 0x7f4d04072df0 con 0x7f4d0406af00 2023-11-04T05:29:21.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d027fc700 1 -- 172.21.15.3:0/1355781615 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040cdce0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d027fc700 1 --2- 172.21.15.3:0/1355781615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040728b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009820 tx=0x7f4cf4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccbcb909c146a1b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:21.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d01ffb700 1 -- 172.21.15.3:0/1355781615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4005e30 con 0x7f4d040cc7a0 2023-11-04T05:29:21.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d01ffb700 1 -- 172.21.15.3:0/1355781615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf4004030 con 0x7f4d040cc7a0 2023-11-04T05:29:21.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d01ffb700 1 -- 172.21.15.3:0/1355781615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40043c0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d09765700 1 -- 172.21.15.3:0/1355781615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 msgr2=0x7f4d040728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d09765700 1 --2- 172.21.15.3:0/1355781615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040728b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009820 tx=0x7f4cf4009b30 comp rx=0 tx=0).stop 2023-11-04T05:29:21.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d09765700 1 -- 172.21.15.3:0/1355781615 shutdown_connections 2023-11-04T05:29:21.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d09765700 1 --2- 172.21.15.3:0/1355781615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040728b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.603+0000 7f4d09765700 1 -- 172.21.15.3:0/1355781615 >> 172.21.15.3:0/1355781615 conn(0x7f4d0406df90 msgr2=0x7f4d0406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:21.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.604+0000 7f4d09765700 1 -- 172.21.15.3:0/1355781615 shutdown_connections 2023-11-04T05:29:21.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.604+0000 7f4d09765700 1 -- 172.21.15.3:0/1355781615 wait complete. 2023-11-04T05:29:21.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.605+0000 7f4d09765700 1 Processor -- start 2023-11-04T05:29:21.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.605+0000 7f4d09765700 1 -- start start 2023-11-04T05:29:21.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d0406ab20 0x7f4d040af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:21.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0406af00 0x7f4d040c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:21.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:21.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c6ef0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c5910 con 0x7f4d0406ab20 2023-11-04T05:29:21.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c5bf0 con 0x7f4d0406af00 2023-11-04T05:29:21.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:21.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53292/0 (socket says 172.21.15.3:53292) 2023-11-04T05:29:21.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 -- 172.21.15.3:0/1490563387 learned_addr learned my addr 172.21.15.3:0/1490563387 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:21.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d027fc700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0406af00 0x7f4d040c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d02ffd700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d0406ab20 0x7f4d040af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 -- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0406af00 msgr2=0x7f4d040c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0406af00 0x7f4d040c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 -- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d0406ab20 msgr2=0x7f4d040af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d0406ab20 0x7f4d040af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 -- 172.21.15.3:0/1490563387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4009430 con 0x7f4d040cc7a0 2023-11-04T05:29:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d037fe700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040c69b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00cca0 tx=0x7f4cfc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00cea0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00ade0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d040c5ed0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc007310 con 0x7f4d040cc7a0 2023-11-04T05:29:21.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.606+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d040c6490 con 0x7f4d040cc7a0 2023-11-04T05:29:21.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.607+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040c66b0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.609+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4cfc007470 con 0x7f4d040cc7a0 2023-11-04T05:29:21.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.609+0000 7f4ceb7fe700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4cf006af00 0x7f4cf006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.609+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4cfc089af0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.609+0000 7f4d02ffd700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4cf006af00 0x7f4cf006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.610+0000 7f4d02ffd700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4cf006af00 0x7f4cf006d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8004f30 tx=0x7f4cf8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:21.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.613+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4cfc054950 con 0x7f4d040cc7a0 2023-11-04T05:29:21.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.814+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d04074df0 con 0x7f4d040cc7a0 2023-11-04T05:29:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.815+0000 7f4ceb7fe700 1 -- 172.21.15.3:0/1490563387 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7f4cfc018090 con 0x7f4d040cc7a0 2023-11-04T05:29:21.817 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:21.817 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:21.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.817+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4cf006af00 msgr2=0x7f4cf006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.817+0000 7f4d09765700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4cf006af00 0x7f4cf006d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8004f30 tx=0x7f4cf8009040 comp rx=0 tx=0).stop 2023-11-04T05:29:21.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.817+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 msgr2=0x7f4d040c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:21.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.817+0000 7f4d09765700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040c69b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00cca0 tx=0x7f4cfc00a340 comp rx=0 tx=0).stop 2023-11-04T05:29:21.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.817+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 shutdown_connections 2023-11-04T05:29:21.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4cf006af00 0x7f4cf006d3c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d0406ab20 0x7f4d040af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d040cc7a0 0x7f4d040c69b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 --2- 172.21.15.3:0/1490563387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0406af00 0x7f4d040c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 >> 172.21.15.3:0/1490563387 conn(0x7f4d0406df90 msgr2=0x7f4d04159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 shutdown_connections 2023-11-04T05:29:21.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.818+0000 7f4d09765700 1 -- 172.21.15.3:0/1490563387 wait complete. 2023-11-04T05:29:21.831 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:21.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.998+0000 7f2835096700 1 Processor -- start 2023-11-04T05:29:21.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.998+0000 7f2835096700 1 -- start start 2023-11-04T05:29:22.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f2835096700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f283015f8b0 0x7f283015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f2835096700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283015c330 con 0x7f28300668e0 2023-11-04T05:29:22.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f2835096700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283015c610 con 0x7f28300cdce0 2023-11-04T05:29:22.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f2835096700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283015ceb0 con 0x7f283015f8b0 2023-11-04T05:29:22.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f283015f8b0 0x7f283015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f28300668e0 0x7f2830066cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58644/0 (socket says 172.21.15.3:58644) 2023-11-04T05:29:22.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f283015f8b0 0x7f283015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53300/0 (socket says 172.21.15.3:53300) 2023-11-04T05:29:22.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282ed9d700 1 -- 172.21.15.3:0/206491583 learned_addr learned my addr 172.21.15.3:0/206491583 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282f59e700 1 -- 172.21.15.3:0/206491583 >> v1:172.21.15.188:6789/0 conn(0x7f28300cdce0 legacy=0x7f283006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282f59e700 1 -- 172.21.15.3:0/206491583 >> v1:172.21.15.3:6789/0 conn(0x7f28300668e0 legacy=0x7f2830066cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282dd9b700 1 -- 172.21.15.3:0/206491583 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138500041 0 0) 0x7f283015c330 con 0x7f28300668e0 2023-11-04T05:29:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282f59e700 1 -- 172.21.15.3:0/206491583 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283015c8f0 con 0x7f283015f8b0 2023-11-04T05:29:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282f59e700 1 --2- 172.21.15.3:0/206491583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283015f8b0 0x7f283015fd60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2828009d90 tx=0x7f282800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8fb517bf37bd11 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282dd9b700 1 -- 172.21.15.3:0/206491583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800e070 con 0x7f283015f8b0 2023-11-04T05:29:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282dd9b700 1 -- 172.21.15.3:0/206491583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800bbd0 con 0x7f283015f8b0 2023-11-04T05:29:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:21.999+0000 7f282dd9b700 1 -- 172.21.15.3:0/206491583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828012430 con 0x7f283015f8b0 2023-11-04T05:29:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 -- 172.21.15.3:0/206491583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283015f8b0 msgr2=0x7f283015fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 --2- 172.21.15.3:0/206491583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283015f8b0 0x7f283015fd60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2828009d90 tx=0x7f282800b330 comp rx=0 tx=0).stop 2023-11-04T05:29:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 -- 172.21.15.3:0/206491583 shutdown_connections 2023-11-04T05:29:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 --2- 172.21.15.3:0/206491583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283015f8b0 0x7f283015fd60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 -- 172.21.15.3:0/206491583 >> 172.21.15.3:0/206491583 conn(0x7f28300cc860 msgr2=0x7f28300ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 -- 172.21.15.3:0/206491583 shutdown_connections 2023-11-04T05:29:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.000+0000 7f2835096700 1 -- 172.21.15.3:0/206491583 wait complete. 2023-11-04T05:29:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.002+0000 7f2835096700 1 Processor -- start 2023-11-04T05:29:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.002+0000 7f2835096700 1 -- start start 2023-11-04T05:29:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f2835096700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 0x7f28301f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f2835096700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28300cdce0 0x7f283006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f2835096700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f283015f8b0 0x7f28301f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f2835096700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28301467e0 con 0x7f28300668e0 2023-11-04T05:29:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f2835096700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28301f57d0 con 0x7f283015f8b0 2023-11-04T05:29:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f2835096700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2830142640 con 0x7f28300cdce0 2023-11-04T05:29:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 0x7f28301f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 0x7f28301f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53302/0 (socket says 172.21.15.3:53302) 2023-11-04T05:29:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 -- 172.21.15.3:0/1175155078 learned_addr learned my addr 172.21.15.3:0/1175155078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 -- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28300cdce0 msgr2=0x7f283006b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282f59e700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f283015f8b0 0x7f28301f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282e59c700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28300cdce0 0x7f283006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28300cdce0 0x7f283006b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 -- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f283015f8b0 msgr2=0x7f28301f0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f283015f8b0 0x7f28301f0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 -- 172.21.15.3:0/1175155078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2828009a70 con 0x7f28300668e0 2023-11-04T05:29:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f282ed9d700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 0x7f28301f48c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f282400c520 tx=0x7f282400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.003+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004c80 con 0x7f28300668e0 2023-11-04T05:29:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.004+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28301f11c0 con 0x7f28300668e0 2023-11-04T05:29:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.004+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2824004e00 con 0x7f28300668e0 2023-11-04T05:29:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.004+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824007310 con 0x7f28300668e0 2023-11-04T05:29:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.004+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28301f5210 con 0x7f28300668e0 2023-11-04T05:29:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.005+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28300cfb30 con 0x7f28300668e0 2023-11-04T05:29:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.005+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2824007470 con 0x7f28300668e0 2023-11-04T05:29:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.005+0000 7f28177fe700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f281c06af00 0x7f281c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.006+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2824089700 con 0x7f28300668e0 2023-11-04T05:29:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.006+0000 7f282e59c700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f281c06af00 0x7f281c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.006+0000 7f282e59c700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f281c06af00 0x7f281c06d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28300b1920 tx=0x7f2820009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.010+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f2824054400 con 0x7f28300668e0 2023-11-04T05:29:22.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.214+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28300008d0 con 0x7f28300668e0 2023-11-04T05:29:22.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.215+0000 7f28177fe700 1 -- 172.21.15.3:0/1175155078 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2824057a20 con 0x7f28300668e0 2023-11-04T05:29:22.217 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:22.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f281c06af00 msgr2=0x7f281c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f281c06af00 0x7f281c06d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28300b1920 tx=0x7f2820009040 comp rx=0 tx=0).stop 2023-11-04T05:29:22.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 msgr2=0x7f28301f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 0x7f28301f48c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f282400c520 tx=0x7f282400c8e0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 shutdown_connections 2023-11-04T05:29:22.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f281c06af00 0x7f281c06d3c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f283015f8b0 0x7f28301f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28300668e0 0x7f28301f48c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 --2- 172.21.15.3:0/1175155078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28300cdce0 0x7f283006b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 >> 172.21.15.3:0/1175155078 conn(0x7f28300cc860 msgr2=0x7f2830066cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:22.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 shutdown_connections 2023-11-04T05:29:22.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.217+0000 7f2835096700 1 -- 172.21.15.3:0/1175155078 wait complete. 2023-11-04T05:29:22.231 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:22.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.359+0000 7f1a3df3e700 1 Processor -- start 2023-11-04T05:29:22.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.359+0000 7f1a3df3e700 1 -- start start 2023-11-04T05:29:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a3df3e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a3806ab20 0x7f1a3806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a3df3e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a380d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a3df3e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a380caca0 con 0x7f1a381611d0 2023-11-04T05:29:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a3df3e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a380ce0a0 con 0x7f1a3806ab20 2023-11-04T05:29:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a3df3e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3806b4e0 con 0x7f1a380d02d0 2023-11-04T05:29:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a377fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1a381611d0 0x7f1a380ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58658/0 (socket says 172.21.15.3:58658) 2023-11-04T05:29:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a377fe700 1 -- 172.21.15.3:0/3796158035 learned_addr learned my addr 172.21.15.3:0/3796158035 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:22.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a37fff700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a380d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/3796158035 >> v2:172.21.15.188:3300/0 conn(0x7f1a3806ab20 0x7f1a3806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537156867 0 0) 0x7f1a380caca0 con 0x7f1a381611d0 2023-11-04T05:29:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a200035b0 con 0x7f1a381611d0 2023-11-04T05:29:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930038121 0 0) 0x7f1a200035b0 con 0x7f1a381611d0 2023-11-04T05:29:22.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 msgr2=0x7f1a380d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a380d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 >> v2:172.21.15.188:3300/0 conn(0x7f1a3806ab20 msgr2=0x7f1a3806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 --2- 172.21.15.3:0/3796158035 >> v2:172.21.15.188:3300/0 conn(0x7f1a3806ab20 0x7f1a3806afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a380cfe40 con 0x7f1a381611d0 2023-11-04T05:29:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f1a28001670 con 0x7f1a381611d0 2023-11-04T05:29:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 >> v1:172.21.15.3:6789/0 conn(0x7f1a381611d0 legacy=0x7f1a380ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a20003a00 0x7f1a20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a20004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a20005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380ce0a0 con 0x7f1a20004850 2023-11-04T05:29:22.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3806b4e0 con 0x7f1a20003a00 2023-11-04T05:29:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.360+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2000ba10 con 0x7f1a380d02d0 2023-11-04T05:29:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a20004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a377fe700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a20003a00 0x7f1a20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a37fff700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a20005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 -- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 msgr2=0x7f1a20005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:22.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a20005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 -- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a20003a00 msgr2=0x7f1a20003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a20003a00 0x7f1a20003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 -- 172.21.15.3:0/3796158035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200035b0 con 0x7f1a20004850 2023-11-04T05:29:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a37fff700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380d02d0 0x7f1a20005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:22.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a20004cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009b60 tx=0x7f1a20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdbe3bf9b1c2c5e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00e6c0 con 0x7f1a20004850 2023-11-04T05:29:22.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00e820 con 0x7f1a20004850 2023-11-04T05:29:22.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.361+0000 7f1a367fc700 1 -- 172.21.15.3:0/3796158035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00eb90 con 0x7f1a20004850 2023-11-04T05:29:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a3df3e700 1 -- 172.21.15.3:0/3796158035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 msgr2=0x7f1a20004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/3796158035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a20004cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009b60 tx=0x7f1a20005880 comp rx=0 tx=0).stop 2023-11-04T05:29:22.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a37fff700 1 -- 172.21.15.3:0/3796158035 reap_dead start 2023-11-04T05:29:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a3df3e700 1 -- 172.21.15.3:0/3796158035 shutdown_connections 2023-11-04T05:29:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a3df3e700 1 -- 172.21.15.3:0/3796158035 >> 172.21.15.3:0/3796158035 conn(0x7f1a38072560 msgr2=0x7f1a38099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:22.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a3df3e700 1 -- 172.21.15.3:0/3796158035 shutdown_connections 2023-11-04T05:29:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.362+0000 7f1a3df3e700 1 -- 172.21.15.3:0/3796158035 wait complete. 2023-11-04T05:29:22.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.363+0000 7f1a3df3e700 1 Processor -- start 2023-11-04T05:29:22.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.363+0000 7f1a3df3e700 1 -- start start 2023-11-04T05:29:22.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a3df3e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a38153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a3df3e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380c5390 0x7f1a380c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a3df3e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a380c5810 0x7f1a380c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a3df3e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380c6a60 con 0x7f1a20004850 2023-11-04T05:29:22.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a3df3e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380c6e80 con 0x7f1a380c5810 2023-11-04T05:29:22.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a3df3e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380c5de0 con 0x7f1a380c5390 2023-11-04T05:29:22.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a38153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a38153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53322/0 (socket says 172.21.15.3:53322) 2023-11-04T05:29:22.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 -- 172.21.15.3:0/1912023606 learned_addr learned my addr 172.21.15.3:0/1912023606 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:22.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 -- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380c5390 msgr2=0x7f1a380c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380c5390 0x7f1a380c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 -- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a380c5810 msgr2=0x7f1a380c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a380c5810 0x7f1a380c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 -- 172.21.15.3:0/1912023606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a380c60c0 con 0x7f1a20004850 2023-11-04T05:29:22.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.364+0000 7f1a377fe700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a38153dd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1a28007340 tx=0x7f1a28007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.365+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800f640 con 0x7f1a20004850 2023-11-04T05:29:22.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.365+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2800f7a0 con 0x7f1a20004850 2023-11-04T05:29:22.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.365+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a380c63a0 con 0x7f1a20004850 2023-11-04T05:29:22.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.365+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800fb50 con 0x7f1a20004850 2023-11-04T05:29:22.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.365+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a38154310 con 0x7f1a20004850 2023-11-04T05:29:22.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.365+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a380cf3f0 con 0x7f1a20004850 2023-11-04T05:29:22.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.366+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a2800fcb0 con 0x7f1a20004850 2023-11-04T05:29:22.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.366+0000 7f1a1ffff700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1a2406af00 0x7f1a2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.367+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1a28089910 con 0x7f1a20004850 2023-11-04T05:29:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.367+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1a2406af00 0x7f1a2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.367+0000 7f1a36ffd700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1a2406af00 0x7f1a2406d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a380c71a0 tx=0x7f1a2c013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.371+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1a28054600 con 0x7f1a20004850 2023-11-04T05:29:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.572+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a3806fa20 con 0x7f1a20004850 2023-11-04T05:29:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.572+0000 7f1a1ffff700 1 -- 172.21.15.3:0/1912023606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a28057c20 con 0x7f1a20004850 2023-11-04T05:29:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.574+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1a2406af00 msgr2=0x7f1a2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.574+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1a2406af00 0x7f1a2406d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a380c71a0 tx=0x7f1a2c013000 comp rx=0 tx=0).stop 2023-11-04T05:29:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.574+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 msgr2=0x7f1a38153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.574+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a38153dd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1a28007340 tx=0x7f1a28007650 comp rx=0 tx=0).stop 2023-11-04T05:29:22.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 shutdown_connections 2023-11-04T05:29:22.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1a2406af00 0x7f1a2406d3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a380c5810 0x7f1a380c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a20004850 0x7f1a38153dd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 --2- 172.21.15.3:0/1912023606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a380c5390 0x7f1a380c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 >> 172.21.15.3:0/1912023606 conn(0x7f1a38072560 msgr2=0x7f1a3806e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:22.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 shutdown_connections 2023-11-04T05:29:22.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.575+0000 7f1a3df3e700 1 -- 172.21.15.3:0/1912023606 wait complete. 2023-11-04T05:29:22.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:22.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.717+0000 7f8c348be700 1 Processor -- start 2023-11-04T05:29:22.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.717+0000 7f8c348be700 1 -- start start 2023-11-04T05:29:22.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.718+0000 7f8c348be700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c300ceaa0 0x7f8c300cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.718+0000 7f8c348be700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300cee80 0x7f8c3015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.718+0000 7f8c348be700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c300ccc70 con 0x7f8c3015b260 2023-11-04T05:29:22.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.718+0000 7f8c348be700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300cdee0 con 0x7f8c300ceaa0 2023-11-04T05:29:22.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.718+0000 7f8c348be700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3015baa0 con 0x7f8c300cee80 2023-11-04T05:29:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8c3015b260 0x7f8c300caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58680/0 (socket says 172.21.15.3:58680) 2023-11-04T05:29:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2ed9d700 1 -- 172.21.15.3:0/1962781718 learned_addr learned my addr 172.21.15.3:0/1962781718 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2dd9b700 1 --2- 172.21.15.3:0/1962781718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300cee80 0x7f8c3015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2e59c700 1 --2- 172.21.15.3:0/1962781718 >> v2:172.21.15.188:3300/0 conn(0x7f8c300ceaa0 0x7f8c300cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2d59a700 1 -- 172.21.15.3:0/1962781718 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522158727 0 0) 0x7f8c300ccc70 con 0x7f8c3015b260 2023-11-04T05:29:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2d59a700 1 -- 172.21.15.3:0/1962781718 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c1c0035b0 con 0x7f8c3015b260 2023-11-04T05:29:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2d59a700 1 -- 172.21.15.3:0/1962781718 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422905900 0 0) 0x7f8c1c0035b0 con 0x7f8c3015b260 2023-11-04T05:29:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2dd9b700 1 -- 172.21.15.3:0/1962781718 >> v2:172.21.15.188:3300/0 conn(0x7f8c300ceaa0 msgr2=0x7f8c300cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2dd9b700 1 --2- 172.21.15.3:0/1962781718 >> v2:172.21.15.188:3300/0 conn(0x7f8c300ceaa0 0x7f8c300cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2dd9b700 1 -- 172.21.15.3:0/1962781718 >> v1:172.21.15.3:6789/0 conn(0x7f8c3015b260 legacy=0x7f8c300caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2dd9b700 1 -- 172.21.15.3:0/1962781718 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300ccec0 con 0x7f8c300cee80 2023-11-04T05:29:22.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2dd9b700 1 --2- 172.21.15.3:0/1962781718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300cee80 0x7f8c3015ad20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009840 tx=0x7f8c20009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ee27018acfc14d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.719+0000 7f8c2d59a700 1 -- 172.21.15.3:0/1962781718 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c280019f0 con 0x7f8c300cee80 2023-11-04T05:29:22.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c2d59a700 1 -- 172.21.15.3:0/1962781718 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20004030 con 0x7f8c300cee80 2023-11-04T05:29:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c2d59a700 1 -- 172.21.15.3:0/1962781718 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200043c0 con 0x7f8c300cee80 2023-11-04T05:29:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 -- 172.21.15.3:0/1962781718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300cee80 msgr2=0x7f8c3015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 --2- 172.21.15.3:0/1962781718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300cee80 0x7f8c3015ad20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009840 tx=0x7f8c20009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 -- 172.21.15.3:0/1962781718 shutdown_connections 2023-11-04T05:29:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 --2- 172.21.15.3:0/1962781718 >> v2:172.21.15.188:3300/0 conn(0x7f8c300ceaa0 0x7f8c300cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 --2- 172.21.15.3:0/1962781718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300cee80 0x7f8c3015ad20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 -- 172.21.15.3:0/1962781718 >> 172.21.15.3:0/1962781718 conn(0x7f8c30072560 msgr2=0x7f8c30099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:22.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 -- 172.21.15.3:0/1962781718 shutdown_connections 2023-11-04T05:29:22.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.720+0000 7f8c348be700 1 -- 172.21.15.3:0/1962781718 wait complete. 2023-11-04T05:29:22.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 Processor -- start 2023-11-04T05:29:22.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 -- start start 2023-11-04T05:29:22.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300ceaa0 0x7f8c300b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c300cee80 0x7f8c301571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 0x7f8c300c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301f67d0 con 0x7f8c3015b260 2023-11-04T05:29:22.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301f69e0 con 0x7f8c300cee80 2023-11-04T05:29:22.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c348be700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301f6cc0 con 0x7f8c300ceaa0 2023-11-04T05:29:22.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c2ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 0x7f8c300c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c2ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 0x7f8c300c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53338/0 (socket says 172.21.15.3:53338) 2023-11-04T05:29:22.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.722+0000 7f8c2ed9d700 1 -- 172.21.15.3:0/1968598701 learned_addr learned my addr 172.21.15.3:0/1968598701 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2dd9b700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c300cee80 0x7f8c301571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2e59c700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300ceaa0 0x7f8c300b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2ed9d700 1 -- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300ceaa0 msgr2=0x7f8c300b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2ed9d700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300ceaa0 0x7f8c300b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2ed9d700 1 -- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c300cee80 msgr2=0x7f8c301571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2ed9d700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c300cee80 0x7f8c301571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2ed9d700 1 -- 172.21.15.3:0/1968598701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200094a0 con 0x7f8c3015b260 2023-11-04T05:29:22.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2e59c700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300ceaa0 0x7f8c300b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:22.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c2ed9d700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 0x7f8c300c9c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cc00 tx=0x7f8c2800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800ab30 con 0x7f8c3015b260 2023-11-04T05:29:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800acb0 con 0x7f8c3015b260 2023-11-04T05:29:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c301f6fa0 con 0x7f8c3015b260 2023-11-04T05:29:22.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28007310 con 0x7f8c3015b260 2023-11-04T05:29:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.723+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c301f7560 con 0x7f8c3015b260 2023-11-04T05:29:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.724+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30159cd0 con 0x7f8c3015b260 2023-11-04T05:29:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.724+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c28007470 con 0x7f8c3015b260 2023-11-04T05:29:22.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.725+0000 7f8c26ffd700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c1006af00 0x7f8c1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:22.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.725+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8c28089840 con 0x7f8c3015b260 2023-11-04T05:29:22.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.725+0000 7f8c2e59c700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c1006af00 0x7f8c1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:22.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.725+0000 7f8c2e59c700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c1006af00 0x7f8c1006d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c300cb7f0 tx=0x7f8c18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:22.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.729+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8c28054530 con 0x7f8c3015b260 2023-11-04T05:29:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.935+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c30074df0 con 0x7f8c3015b260 2023-11-04T05:29:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.935+0000 7f8c26ffd700 1 -- 172.21.15.3:0/1968598701 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c280180a0 con 0x7f8c3015b260 2023-11-04T05:29:22.937 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:22.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c1006af00 msgr2=0x7f8c1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c1006af00 0x7f8c1006d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c300cb7f0 tx=0x7f8c18008040 comp rx=0 tx=0).stop 2023-11-04T05:29:22.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 msgr2=0x7f8c300c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:22.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 0x7f8c300c9c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cc00 tx=0x7f8c2800cf10 comp rx=0 tx=0).stop 2023-11-04T05:29:22.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 shutdown_connections 2023-11-04T05:29:22.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8c1006af00 0x7f8c1006d3c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3015b260 0x7f8c300c9c20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c300ceaa0 0x7f8c300b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 --2- 172.21.15.3:0/1968598701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c300cee80 0x7f8c301571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:22.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 >> 172.21.15.3:0/1968598701 conn(0x7f8c30072560 msgr2=0x7f8c300738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:22.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 shutdown_connections 2023-11-04T05:29:22.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:22.937+0000 7f8c348be700 1 -- 172.21.15.3:0/1968598701 wait complete. 2023-11-04T05:29:22.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.078+0000 7f4d41c1e700 1 Processor -- start 2023-11-04T05:29:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.078+0000 7f4d41c1e700 1 -- start start 2023-11-04T05:29:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d41c1e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d3c0cd1d0 0x7f4d3c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d41c1e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0cd9b0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d41c1e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0cdc90 con 0x7f4d3c0b25d0 2023-11-04T05:29:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d41c1e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0cdf70 con 0x7f4d3c0cd1d0 2023-11-04T05:29:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d3affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d3c0cd1d0 0x7f4d3c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d3bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d3c0b29b0 0x7f4d3c0b0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58694/0 (socket says 172.21.15.3:58694) 2023-11-04T05:29:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d3bfff700 1 -- 172.21.15.3:0/1534920241 learned_addr learned my addr 172.21.15.3:0/1534920241 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.079+0000 7f4d3affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d3c0cd1d0 0x7f4d3c0cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53344/0 (socket says 172.21.15.3:53344) 2023-11-04T05:29:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3affd700 1 -- 172.21.15.3:0/1534920241 >> v1:172.21.15.188:6789/0 conn(0x7f4d3c0b25d0 legacy=0x7f4d3c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3affd700 1 -- 172.21.15.3:0/1534920241 >> v1:172.21.15.3:6789/0 conn(0x7f4d3c0b29b0 legacy=0x7f4d3c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3affd700 1 -- 172.21.15.3:0/1534920241 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0ce250 con 0x7f4d3c0cd1d0 2023-11-04T05:29:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3a7fc700 1 -- 172.21.15.3:0/1534920241 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636255031 0 0) 0x7f4d3c0cd9b0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3affd700 1 --2- 172.21.15.3:0/1534920241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0cc1d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009870 tx=0x7f4d2c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe81db9b84ede4a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3a7fc700 1 -- 172.21.15.3:0/1534920241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c004230 con 0x7f4d3c0cd1d0 2023-11-04T05:29:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3a7fc700 1 -- 172.21.15.3:0/1534920241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c004390 con 0x7f4d3c0cd1d0 2023-11-04T05:29:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d3a7fc700 1 -- 172.21.15.3:0/1534920241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0036a0 con 0x7f4d3c0cd1d0 2023-11-04T05:29:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d41c1e700 1 -- 172.21.15.3:0/1534920241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0cd1d0 msgr2=0x7f4d3c0cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/1534920241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0cc1d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009870 tx=0x7f4d2c009b80 comp rx=0 tx=0).stop 2023-11-04T05:29:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d41c1e700 1 -- 172.21.15.3:0/1534920241 shutdown_connections 2023-11-04T05:29:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/1534920241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0cc1d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.080+0000 7f4d41c1e700 1 -- 172.21.15.3:0/1534920241 >> 172.21.15.3:0/1534920241 conn(0x7f4d3c0d1630 msgr2=0x7f4d3c0d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.081+0000 7f4d41c1e700 1 -- 172.21.15.3:0/1534920241 shutdown_connections 2023-11-04T05:29:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.081+0000 7f4d41c1e700 1 -- 172.21.15.3:0/1534920241 wait complete. 2023-11-04T05:29:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.082+0000 7f4d41c1e700 1 Processor -- start 2023-11-04T05:29:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.082+0000 7f4d41c1e700 1 -- start start 2023-11-04T05:29:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0b25d0 0x7f4d3c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 0x7f4d3c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b41f0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b4d30 con 0x7f4d3c0b25d0 2023-11-04T05:29:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b4400 con 0x7f4d3c0cd1d0 2023-11-04T05:29:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 0x7f4d3c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 0x7f4d3c0b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53350/0 (socket says 172.21.15.3:53350) 2023-11-04T05:29:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 -- 172.21.15.3:0/4266727188 learned_addr learned my addr 172.21.15.3:0/4266727188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3bfff700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3b7fe700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0b25d0 0x7f4d3c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 -- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0cd1d0 msgr2=0x7f4d3c0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 -- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0b25d0 msgr2=0x7f4d3c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0b25d0 0x7f4d3c0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 -- 172.21.15.3:0/4266727188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0094a0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3bfff700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3b7fe700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0b25d0 0x7f4d3c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d3affd700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 0x7f4d3c0b5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009fd0 tx=0x7f4d2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c011830 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c01f070 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c011b90 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.083+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c0b46e0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.084+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0ca830 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.084+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c0ca9c0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.085+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d2c011cf0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.086+0000 7f4d23fff700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4d2806af00 0x7f4d2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.086+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d2c021070 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.086+0000 7f4d3b7fe700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4d2806af00 0x7f4d2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.086+0000 7f4d3b7fe700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4d2806af00 0x7f4d2806d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006fd0 tx=0x7f4d30008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:23.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.090+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4d2c0565a0 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.291+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4d3c0d1430 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.291+0000 7f4d23fff700 1 -- 172.21.15.3:0/4266727188 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4d2c02b020 con 0x7f4d3c0b29b0 2023-11-04T05:29:23.293 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:23.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4d2806af00 msgr2=0x7f4d2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4d2806af00 0x7f4d2806d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006fd0 tx=0x7f4d30008040 comp rx=0 tx=0).stop 2023-11-04T05:29:23.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 msgr2=0x7f4d3c0b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 0x7f4d3c0b5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009fd0 tx=0x7f4d2c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:23.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 shutdown_connections 2023-11-04T05:29:23.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4d2806af00 0x7f4d2806d3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d3c0b25d0 0x7f4d3c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3c0cd1d0 0x7f4d3c0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 --2- 172.21.15.3:0/4266727188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d3c0b29b0 0x7f4d3c0b5700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 >> 172.21.15.3:0/4266727188 conn(0x7f4d3c0d1630 msgr2=0x7f4d3c15cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:23.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 shutdown_connections 2023-11-04T05:29:23.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.294+0000 7f4d41c1e700 1 -- 172.21.15.3:0/4266727188 wait complete. 2023-11-04T05:29:23.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T05:29:23.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.433+0000 7f87b2303700 1 Processor -- start 2023-11-04T05:29:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.433+0000 7f87b2303700 1 -- start start 2023-11-04T05:29:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b2303700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b2303700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87ac06ab20 0x7f87ac06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b2303700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac0cde30 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b2303700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac06b510 con 0x7f87ac06ab20 2023-11-04T05:29:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b2303700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0cae70 con 0x7f87ac15b910 2023-11-04T05:29:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b08a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87ac06ab20 0x7f87ac06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b08a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87ac06ab20 0x7f87ac06afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53360/0 (socket says 172.21.15.3:53360) 2023-11-04T05:29:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b08a0700 1 -- 172.21.15.3:0/2574065642 learned_addr learned my addr 172.21.15.3:0/2574065642 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87abfff700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.434+0000 7f87b08a0700 1 -- 172.21.15.3:0/2574065642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 msgr2=0x7f87ac15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b08a0700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b08a0700 1 -- 172.21.15.3:0/2574065642 >> v1:172.21.15.188:6789/0 conn(0x7f87ac0c3bf0 legacy=0x7f87ac0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b08a0700 1 -- 172.21.15.3:0/2574065642 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac15c230 con 0x7f87ac06ab20 2023-11-04T05:29:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87abfff700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b08a0700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac06ab20 0x7f87ac06afd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009e50 tx=0x7f87a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acead6e0a7d9bcf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87aaffd700 1 -- 172.21.15.3:0/2574065642 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a400e070 con 0x7f87ac06ab20 2023-11-04T05:29:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87aaffd700 1 -- 172.21.15.3:0/2574065642 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a400b9c0 con 0x7f87ac06ab20 2023-11-04T05:29:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87aaffd700 1 -- 172.21.15.3:0/2574065642 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a400bd70 con 0x7f87ac06ab20 2023-11-04T05:29:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b2303700 1 -- 172.21.15.3:0/2574065642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac06ab20 msgr2=0x7f87ac06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b2303700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac06ab20 0x7f87ac06afd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009e50 tx=0x7f87a400b330 comp rx=0 tx=0).stop 2023-11-04T05:29:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b2303700 1 -- 172.21.15.3:0/2574065642 shutdown_connections 2023-11-04T05:29:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b2303700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac06ab20 0x7f87ac06afd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b2303700 1 --2- 172.21.15.3:0/2574065642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.435+0000 7f87b2303700 1 -- 172.21.15.3:0/2574065642 >> 172.21.15.3:0/2574065642 conn(0x7f87ac0d0560 msgr2=0x7f87ac0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:23.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.436+0000 7f87b2303700 1 -- 172.21.15.3:0/2574065642 shutdown_connections 2023-11-04T05:29:23.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.436+0000 7f87b2303700 1 -- 172.21.15.3:0/2574065642 wait complete. 2023-11-04T05:29:23.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.437+0000 7f87b2303700 1 Processor -- start 2023-11-04T05:29:23.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.437+0000 7f87b2303700 1 -- start start 2023-11-04T05:29:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87ac06ab20 0x7f87ac0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 0x7f87ac0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac1f2380 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0c3ff0 con 0x7f87ac06ab20 2023-11-04T05:29:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac1f15e0 con 0x7f87ac15b910 2023-11-04T05:29:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 0x7f87ac0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 0x7f87ac0b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53364/0 (socket says 172.21.15.3:53364) 2023-11-04T05:29:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 -- 172.21.15.3:0/469272069 learned_addr learned my addr 172.21.15.3:0/469272069 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87abfff700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87ac06ab20 0x7f87ac0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b08a0700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 -- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 msgr2=0x7f87ac0b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac0b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 -- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87ac06ab20 msgr2=0x7f87ac0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87ac06ab20 0x7f87ac0b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 -- 172.21.15.3:0/469272069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4009b30 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87ab7fe700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 0x7f87ac0b4ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f879400c4e0 tx=0x7f879400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940052d0 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794005430 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940038c0 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.438+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac1f18c0 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.439+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac1f1e80 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.440+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac074df0 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.440+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8794003a20 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.440+0000 7f87a8ff9700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f878c06af00 0x7f878c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.440+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8794088d20 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.440+0000 7f87abfff700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f878c06af00 0x7f878c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.441+0000 7f87abfff700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f878c06af00 0x7f878c06d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f879c00c960 tx=0x7f879c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.445+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8794053a10 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.663+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f87ac0b63e0 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.664+0000 7f87a8ff9700 1 -- 172.21.15.3:0/469272069 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8794017080 con 0x7f87ac0c3bf0 2023-11-04T05:29:23.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f878c06af00 msgr2=0x7f878c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f878c06af00 0x7f878c06d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f879c00c960 tx=0x7f879c008040 comp rx=0 tx=0).stop 2023-11-04T05:29:23.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 msgr2=0x7f87ac0b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:23.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 0x7f87ac0b4ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f879400c4e0 tx=0x7f879400c7f0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 shutdown_connections 2023-11-04T05:29:23.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f878c06af00 0x7f878c06d3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87ac06ab20 0x7f87ac0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87ac15b910 0x7f87ac0b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 --2- 172.21.15.3:0/469272069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0c3bf0 0x7f87ac0b4ae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:23.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 >> 172.21.15.3:0/469272069 conn(0x7f87ac0d0560 msgr2=0x7f87ac0cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:23.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 shutdown_connections 2023-11-04T05:29:23.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:23.666+0000 7f87b2303700 1 -- 172.21.15.3:0/469272069 wait complete. 2023-11-04T05:29:23.670 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T05:29:23.679 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-04T05:29:23.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:23.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-04T05:29:23.715 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T05:29:23.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:23.716 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-04T05:29:23.773 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:23.774 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:23.795 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:23.795 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:23.857 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T05:29:23.858 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:23.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:24.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.042+0000 7f121947c700 1 Processor -- start 2023-11-04T05:29:24.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.042+0000 7f121947c700 1 -- start start 2023-11-04T05:29:24.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f121947c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f121415dc40 0x7f121406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f121947c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f12140cd7e0 0x7f12140cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f121947c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121406d680 con 0x7f12140d2200 2023-11-04T05:29:24.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f121947c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12140d1510 con 0x7f121415dc40 2023-11-04T05:29:24.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f121947c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121406d960 con 0x7f12140cd7e0 2023-11-04T05:29:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1212ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f121415dc40 0x7f121406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f12137fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12140d2200 0x7f12140cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58722/0 (socket says 172.21.15.3:58722) 2023-11-04T05:29:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f12137fe700 1 -- 172.21.15.3:0/3445838984 learned_addr learned my addr 172.21.15.3:0/3445838984 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1212ffd700 1 -- 172.21.15.3:0/3445838984 >> v2:172.21.15.188:3300/0 conn(0x7f12140cd7e0 msgr2=0x7f12140cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f12127fc700 1 --2- 172.21.15.3:0/3445838984 >> v2:172.21.15.188:3300/0 conn(0x7f12140cd7e0 0x7f12140cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1212ffd700 1 --2- 172.21.15.3:0/3445838984 >> v2:172.21.15.188:3300/0 conn(0x7f12140cd7e0 0x7f12140cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1212ffd700 1 -- 172.21.15.3:0/3445838984 >> v1:172.21.15.3:6789/0 conn(0x7f12140d2200 legacy=0x7f12140cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1211ffb700 1 -- 172.21.15.3:0/3445838984 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432102941 0 0) 0x7f121406d680 con 0x7f12140d2200 2023-11-04T05:29:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1212ffd700 1 -- 172.21.15.3:0/3445838984 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140d1720 con 0x7f121415dc40 2023-11-04T05:29:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f12127fc700 1 --2- 172.21.15.3:0/3445838984 >> v2:172.21.15.188:3300/0 conn(0x7f12140cd7e0 0x7f12140cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.043+0000 7f1212ffd700 1 --2- 172.21.15.3:0/3445838984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121415dc40 0x7f121406cad0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12140d26f0 tx=0x7f1208009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49f7dce737e635e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f1211ffb700 1 -- 172.21.15.3:0/3445838984 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208011830 con 0x7f121415dc40 2023-11-04T05:29:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f1211ffb700 1 -- 172.21.15.3:0/3445838984 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1208011990 con 0x7f121415dc40 2023-11-04T05:29:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f1211ffb700 1 -- 172.21.15.3:0/3445838984 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208011d20 con 0x7f121415dc40 2023-11-04T05:29:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 -- 172.21.15.3:0/3445838984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121415dc40 msgr2=0x7f121406cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 --2- 172.21.15.3:0/3445838984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121415dc40 0x7f121406cad0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12140d26f0 tx=0x7f1208009b20 comp rx=0 tx=0).stop 2023-11-04T05:29:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 -- 172.21.15.3:0/3445838984 shutdown_connections 2023-11-04T05:29:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 --2- 172.21.15.3:0/3445838984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121415dc40 0x7f121406cad0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 --2- 172.21.15.3:0/3445838984 >> v2:172.21.15.188:3300/0 conn(0x7f12140cd7e0 0x7f12140cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 -- 172.21.15.3:0/3445838984 >> 172.21.15.3:0/3445838984 conn(0x7f121406ab20 msgr2=0x7f121406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 -- 172.21.15.3:0/3445838984 shutdown_connections 2023-11-04T05:29:24.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.044+0000 7f121947c700 1 -- 172.21.15.3:0/3445838984 wait complete. 2023-11-04T05:29:24.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.046+0000 7f121947c700 1 Processor -- start 2023-11-04T05:29:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.046+0000 7f121947c700 1 -- start start 2023-11-04T05:29:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f121947c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140d2200 0x7f12141f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f121947c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415dc40 0x7f12140c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f121947c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 0x7f12140cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f121947c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140c6e00 con 0x7f12140c7d40 2023-11-04T05:29:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f121947c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140c6230 con 0x7f12140d2200 2023-11-04T05:29:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f121947c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140c64d0 con 0x7f121415dc40 2023-11-04T05:29:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 0x7f12140cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 0x7f12140cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53380/0 (socket says 172.21.15.3:53380) 2023-11-04T05:29:24.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 -- 172.21.15.3:0/2322132947 learned_addr learned my addr 172.21.15.3:0/2322132947 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f1212ffd700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140d2200 0x7f12141f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 -- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415dc40 msgr2=0x7f12140c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415dc40 0x7f12140c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 -- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140d2200 msgr2=0x7f12141f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140d2200 0x7f12141f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.047+0000 7f12137fe700 1 -- 172.21.15.3:0/2322132947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080094a0 con 0x7f12140c7d40 2023-11-04T05:29:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.048+0000 7f12137fe700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 0x7f12140cdbd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f120c00cc70 tx=0x7f120c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.049+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00ad80 con 0x7f12140c7d40 2023-11-04T05:29:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.049+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c00ce70 con 0x7f12140c7d40 2023-11-04T05:29:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.049+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12140c67b0 con 0x7f12140c7d40 2023-11-04T05:29:24.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.049+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c0072f0 con 0x7f12140c7d40 2023-11-04T05:29:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.049+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12141f3170 con 0x7f12140c7d40 2023-11-04T05:29:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.050+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12140c6fb0 con 0x7f12140c7d40 2023-11-04T05:29:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.051+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f120c007450 con 0x7f12140c7d40 2023-11-04T05:29:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.051+0000 7f11fb7fe700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f120006af30 0x7f120006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.051+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f120c089820 con 0x7f12140c7d40 2023-11-04T05:29:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.051+0000 7f1212ffd700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f120006af30 0x7f120006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.052+0000 7f1212ffd700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f120006af30 0x7f120006d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12080090b0 tx=0x7f120801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.057+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f120c054510 con 0x7f12140c7d40 2023-11-04T05:29:24.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.260+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1214074df0 con 0x7f12140c7d40 2023-11-04T05:29:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.261+0000 7f11fb7fe700 1 -- 172.21.15.3:0/2322132947 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7f120c018090 con 0x7f12140c7d40 2023-11-04T05:29:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:24.263 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f120006af30 msgr2=0x7f120006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f120006af30 0x7f120006d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12080090b0 tx=0x7f120801c040 comp rx=0 tx=0).stop 2023-11-04T05:29:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 msgr2=0x7f12140cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 0x7f12140cdbd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f120c00cc70 tx=0x7f120c00a340 comp rx=0 tx=0).stop 2023-11-04T05:29:24.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 shutdown_connections 2023-11-04T05:29:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f120006af30 0x7f120006d3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140d2200 0x7f12141f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12140c7d40 0x7f12140cdbd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 --2- 172.21.15.3:0/2322132947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415dc40 0x7f12140c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 >> 172.21.15.3:0/2322132947 conn(0x7f121406ab20 msgr2=0x7f12140b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:24.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 shutdown_connections 2023-11-04T05:29:24.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.263+0000 7f121947c700 1 -- 172.21.15.3:0/2322132947 wait complete. 2023-11-04T05:29:24.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:24.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:24.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 Processor -- start 2023-11-04T05:29:24.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 -- start start 2023-11-04T05:29:24.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67201616e0 0x7f6720161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 0x7f67200b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672006d920 con 0x7f672015dd80 2023-11-04T05:29:24.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67200b3ae0 con 0x7f67201616e0 2023-11-04T05:29:24.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.406+0000 7f6725d20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6720162250 con 0x7f67200b3110 2023-11-04T05:29:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67201616e0 0x7f6720161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671ffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 0x7f67200b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671ffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 0x7f67200b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:44442/0 (socket says 172.21.15.3:44442) 2023-11-04T05:29:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671ffff700 1 -- 172.21.15.3:0/2985443406 learned_addr learned my addr 172.21.15.3:0/2985443406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67201616e0 0x7f6720161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.3:52476/0 (socket says 172.21.15.3:52476) 2023-11-04T05:29:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 -- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 msgr2=0x7f67200b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 --2- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 0x7f67200b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 -- 172.21.15.3:0/2985443406 >> v1:172.21.15.188:6789/0 conn(0x7f672015dd80 legacy=0x7f672006c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671e7fc700 1 -- 172.21.15.3:0/2985443406 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587419024 0 0) 0x7f672006d920 con 0x7f672015dd80 2023-11-04T05:29:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 -- 172.21.15.3:0/2985443406 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672006bee0 con 0x7f67201616e0 2023-11-04T05:29:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671ffff700 1 --2- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 0x7f67200b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.407+0000 7f671effd700 1 --2- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67201616e0 0x7f6720161b60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6714009840 tx=0x7f6714009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4c2b85464aa951 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f671e7fc700 1 -- 172.21.15.3:0/2985443406 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011830 con 0x7f67201616e0 2023-11-04T05:29:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f671e7fc700 1 -- 172.21.15.3:0/2985443406 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714011990 con 0x7f67201616e0 2023-11-04T05:29:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f671e7fc700 1 -- 172.21.15.3:0/2985443406 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011d20 con 0x7f67201616e0 2023-11-04T05:29:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 -- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67201616e0 msgr2=0x7f6720161b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 --2- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67201616e0 0x7f6720161b60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6714009840 tx=0x7f6714009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 -- 172.21.15.3:0/2985443406 shutdown_connections 2023-11-04T05:29:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 --2- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67200b3110 0x7f67200b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 --2- 172.21.15.3:0/2985443406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67201616e0 0x7f6720161b60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 -- 172.21.15.3:0/2985443406 >> 172.21.15.3:0/2985443406 conn(0x7f67200ce110 msgr2=0x7f67200ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 -- 172.21.15.3:0/2985443406 shutdown_connections 2023-11-04T05:29:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.408+0000 7f6725d20700 1 -- 172.21.15.3:0/2985443406 wait complete. 2023-11-04T05:29:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.409+0000 7f6725d20700 1 Processor -- start 2023-11-04T05:29:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 -- start start 2023-11-04T05:29:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 0x7f6720155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67201616e0 0x7f672015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67200b6660 0x7f6720160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200b6240 con 0x7f67200b3110 2023-11-04T05:29:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201562d0 con 0x7f67200b6660 2023-11-04T05:29:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f6725d20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200b52e0 con 0x7f67201616e0 2023-11-04T05:29:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 0x7f6720155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 0x7f6720155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53390/0 (socket says 172.21.15.3:53390) 2023-11-04T05:29:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.410+0000 7f671f7fe700 1 -- 172.21.15.3:0/3046960245 learned_addr learned my addr 172.21.15.3:0/3046960245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671ffff700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67200b6660 0x7f6720160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671effd700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67201616e0 0x7f672015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671f7fe700 1 -- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67201616e0 msgr2=0x7f672015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671f7fe700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67201616e0 0x7f672015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671f7fe700 1 -- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67200b6660 msgr2=0x7f6720160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671f7fe700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67200b6660 0x7f6720160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671f7fe700 1 -- 172.21.15.3:0/3046960245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718009b30 con 0x7f67200b3110 2023-11-04T05:29:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f671f7fe700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 0x7f6720155d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67100011e0 tx=0x7f671000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67100054e0 con 0x7f67200b3110 2023-11-04T05:29:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710003680 con 0x7f67200b3110 2023-11-04T05:29:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67140094a0 con 0x7f67200b3110 2023-11-04T05:29:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710003a30 con 0x7f67200b3110 2023-11-04T05:29:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.411+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67200b58a0 con 0x7f67200b3110 2023-11-04T05:29:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.412+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67200b6380 con 0x7f67200b3110 2023-11-04T05:29:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.413+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6710003b90 con 0x7f67200b3110 2023-11-04T05:29:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.413+0000 7f6707fff700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f670c06af00 0x7f670c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.413+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6710088d10 con 0x7f67200b3110 2023-11-04T05:29:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.413+0000 7f671effd700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f670c06af00 0x7f670c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.414+0000 7f671effd700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f670c06af00 0x7f670c06d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f67140090b0 tx=0x7f6714008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.418+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f67200b6380 con 0x7f67200b3110 2023-11-04T05:29:24.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.618+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67200b6380 con 0x7f67200b3110 2023-11-04T05:29:24.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.618+0000 7f6707fff700 1 -- 172.21.15.3:0/3046960245 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67200b6380 con 0x7f67200b3110 2023-11-04T05:29:24.620 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:24.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f670c06af00 msgr2=0x7f670c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f670c06af00 0x7f670c06d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f67140090b0 tx=0x7f6714008030 comp rx=0 tx=0).stop 2023-11-04T05:29:24.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 msgr2=0x7f6720155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 0x7f6720155d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67100011e0 tx=0x7f671000ca30 comp rx=0 tx=0).stop 2023-11-04T05:29:24.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 shutdown_connections 2023-11-04T05:29:24.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f670c06af00 0x7f670c06d3c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67200b6660 0x7f6720160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67200b3110 0x7f6720155d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 --2- 172.21.15.3:0/3046960245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67201616e0 0x7f672015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.620+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 >> 172.21.15.3:0/3046960245 conn(0x7f67200ce110 msgr2=0x7f672015b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:24.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.621+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 shutdown_connections 2023-11-04T05:29:24.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.621+0000 7f6725d20700 1 -- 172.21.15.3:0/3046960245 wait complete. 2023-11-04T05:29:24.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.761+0000 7f096571d700 1 Processor -- start 2023-11-04T05:29:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.761+0000 7f096571d700 1 -- start start 2023-11-04T05:29:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f096571d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09600d2800 0x7f09600d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f096571d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 0x7f09600d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f096571d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09600d1510 con 0x7f096015db50 2023-11-04T05:29:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f096571d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09600d1760 con 0x7f09600d2800 2023-11-04T05:29:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f096571d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09600d1a40 con 0x7f09600cf790 2023-11-04T05:29:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f095effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f096015db50 0x7f09600cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58746/0 (socket says 172.21.15.3:58746) 2023-11-04T05:29:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f095effd700 1 -- 172.21.15.3:0/2146015032 learned_addr learned my addr 172.21.15.3:0/2146015032 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2146015032 >> v2:172.21.15.188:3300/0 conn(0x7f09600d2800 0x7f09600d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f095f7fe700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 0x7f09600d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055517715 0 0) 0x7f09600d1510 con 0x7f096015db50 2023-11-04T05:29:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.762+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09480035b0 con 0x7f096015db50 2023-11-04T05:29:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4116989908 0 0) 0x7f09480035b0 con 0x7f096015db50 2023-11-04T05:29:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 msgr2=0x7f09600d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 0x7f09600d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 >> v2:172.21.15.188:3300/0 conn(0x7f09600d2800 msgr2=0x7f09600d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 --2- 172.21.15.3:0/2146015032 >> v2:172.21.15.188:3300/0 conn(0x7f09600d2800 0x7f09600d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600d1d20 con 0x7f096015db50 2023-11-04T05:29:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2146015032 >> v2:172.21.15.188:3300/0 conn(0x7f09600d2800 0x7f09600d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f0954001670 con 0x7f096015db50 2023-11-04T05:29:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 >> v1:172.21.15.3:6789/0 conn(0x7f096015db50 legacy=0x7f09600cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 0x7f0948003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0948004850 0x7f0948004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 0x7f0948005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09600d1760 con 0x7f0948004850 2023-11-04T05:29:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09600d1a40 con 0x7f0948003a00 2023-11-04T05:29:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094800ba10 con 0x7f09600cf790 2023-11-04T05:29:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0948004850 0x7f0948004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095effd700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 0x7f0948003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095f7fe700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 0x7f0948005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 -- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 msgr2=0x7f0948005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600cf790 0x7f0948005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 -- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 msgr2=0x7f0948003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 0x7f0948003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 -- 172.21.15.3:0/2146015032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480035b0 con 0x7f0948004850 2023-11-04T05:29:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.763+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0948004850 0x7f0948004cd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0950009d30 tx=0x7f0948005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4df66aa1a1f71dc server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950009e80 con 0x7f0948004850 2023-11-04T05:29:24.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000e9a0 con 0x7f0948004850 2023-11-04T05:29:24.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f095dffb700 1 -- 172.21.15.3:0/2146015032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000ed30 con 0x7f0948004850 2023-11-04T05:29:24.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f096571d700 1 -- 172.21.15.3:0/2146015032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0948004850 msgr2=0x7f0948004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f096571d700 1 --2- 172.21.15.3:0/2146015032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0948004850 0x7f0948004cd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0950009d30 tx=0x7f0948005880 comp rx=0 tx=0).stop 2023-11-04T05:29:24.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f095f7fe700 1 -- 172.21.15.3:0/2146015032 reap_dead start 2023-11-04T05:29:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f096571d700 1 -- 172.21.15.3:0/2146015032 shutdown_connections 2023-11-04T05:29:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f096571d700 1 -- 172.21.15.3:0/2146015032 >> 172.21.15.3:0/2146015032 conn(0x7f096008aff0 msgr2=0x7f09600d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f096571d700 1 -- 172.21.15.3:0/2146015032 shutdown_connections 2023-11-04T05:29:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.764+0000 7f096571d700 1 -- 172.21.15.3:0/2146015032 wait complete. 2023-11-04T05:29:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.766+0000 7f096571d700 1 Processor -- start 2023-11-04T05:29:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.766+0000 7f096571d700 1 -- start start 2023-11-04T05:29:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f096571d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948004850 0x7f09601f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f096571d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 0x7f096015fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f096571d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600b15c0 0x7f09600b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f096571d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09600b1f90 con 0x7f096015e2b0 2023-11-04T05:29:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f096571d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09601f44a0 con 0x7f0948004850 2023-11-04T05:29:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f096571d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096015e740 con 0x7f09600b15c0 2023-11-04T05:29:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 0x7f096015fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 0x7f096015fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53412/0 (socket says 172.21.15.3:53412) 2023-11-04T05:29:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 -- 172.21.15.3:0/2386434229 learned_addr learned my addr 172.21.15.3:0/2386434229 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095f7fe700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600b15c0 0x7f09600b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095effd700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948004850 0x7f09601f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 -- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600b15c0 msgr2=0x7f09600b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600b15c0 0x7f09600b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 -- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948004850 msgr2=0x7f09601f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948004850 0x7f09601f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095e7fc700 1 -- 172.21.15.3:0/2386434229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096015ea20 con 0x7f096015e2b0 2023-11-04T05:29:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.767+0000 7f095effd700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948004850 0x7f09601f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.768+0000 7f095e7fc700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 0x7f096015fdc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f09500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.768+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950003ab0 con 0x7f096015e2b0 2023-11-04T05:29:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.768+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950003c30 con 0x7f096015e2b0 2023-11-04T05:29:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.768+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096015ed00 con 0x7f096015e2b0 2023-11-04T05:29:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.768+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950009e80 con 0x7f096015e2b0 2023-11-04T05:29:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.768+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09601f4670 con 0x7f096015e2b0 2023-11-04T05:29:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.769+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09600d1030 con 0x7f096015e2b0 2023-11-04T05:29:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.769+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0950013470 con 0x7f096015e2b0 2023-11-04T05:29:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.770+0000 7f09477fe700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f094c06af30 0x7f094c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.770+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f095008b600 con 0x7f096015e2b0 2023-11-04T05:29:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.770+0000 7f095effd700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f094c06af30 0x7f094c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.770+0000 7f095effd700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f094c06af30 0x7f094c06d3f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0954000fb0 tx=0x7f0954009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.774+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f09500562e0 con 0x7f096015e2b0 2023-11-04T05:29:24.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.975+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09601f4830 con 0x7f096015e2b0 2023-11-04T05:29:24.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.976+0000 7f09477fe700 1 -- 172.21.15.3:0/2386434229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0950059900 con 0x7f096015e2b0 2023-11-04T05:29:24.978 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f094c06af30 msgr2=0x7f094c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f094c06af30 0x7f094c06d3f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0954000fb0 tx=0x7f0954009040 comp rx=0 tx=0).stop 2023-11-04T05:29:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 msgr2=0x7f096015fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 0x7f096015fdc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f09500047c0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 shutdown_connections 2023-11-04T05:29:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f094c06af30 0x7f094c06d3f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948004850 0x7f09601f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09600b15c0 0x7f09600b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 --2- 172.21.15.3:0/2386434229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f096015e2b0 0x7f096015fdc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 >> 172.21.15.3:0/2386434229 conn(0x7f096008aff0 msgr2=0x7f09600b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 shutdown_connections 2023-11-04T05:29:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:24.978+0000 7f096571d700 1 -- 172.21.15.3:0/2386434229 wait complete. 2023-11-04T05:29:24.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.119+0000 7ff3264c3700 1 Processor -- start 2023-11-04T05:29:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.119+0000 7ff3264c3700 1 -- start start 2023-11-04T05:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff3264c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff320155df0 0x7ff3200cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff3264c3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff32006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff3264c3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200b0f20 con 0x7ff32015be50 2023-11-04T05:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff3264c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32015f4a0 con 0x7ff320155df0 2023-11-04T05:29:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff3264c3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320160370 con 0x7ff32015c2b0 2023-11-04T05:29:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff320155df0 0x7ff3200cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff32015be50 0x7ff32015fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58766/0 (socket says 172.21.15.3:58766) 2023-11-04T05:29:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31f7fe700 1 -- 172.21.15.3:0/1514550053 learned_addr learned my addr 172.21.15.3:0/1514550053 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 --2- 172.21.15.3:0/1514550053 >> v2:172.21.15.3:3300/0 conn(0x7ff320155df0 0x7ff3200cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53418/0 (socket says 172.21.15.3:53418) 2023-11-04T05:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff324a60700 1 --2- 172.21.15.3:0/1514550053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff32006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 -- 172.21.15.3:0/1514550053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 msgr2=0x7ff32006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 --2- 172.21.15.3:0/1514550053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff32006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 -- 172.21.15.3:0/1514550053 >> v1:172.21.15.3:6789/0 conn(0x7ff32015be50 legacy=0x7ff32015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31effd700 1 -- 172.21.15.3:0/1514550053 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041566648 0 0) 0x7ff308001110 con 0x7ff32015be50 2023-11-04T05:29:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 -- 172.21.15.3:0/1514550053 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32015c770 con 0x7ff320155df0 2023-11-04T05:29:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.120+0000 7ff31ffff700 1 --2- 172.21.15.3:0/1514550053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff320155df0 0x7ff3200cc100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff310009840 tx=0x7ff310009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d1a47e33fcdfe server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff31effd700 1 -- 172.21.15.3:0/1514550053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310011830 con 0x7ff320155df0 2023-11-04T05:29:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff31effd700 1 -- 172.21.15.3:0/1514550053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff310011990 con 0x7ff320155df0 2023-11-04T05:29:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff31effd700 1 -- 172.21.15.3:0/1514550053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310011d20 con 0x7ff320155df0 2023-11-04T05:29:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 -- 172.21.15.3:0/1514550053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff320155df0 msgr2=0x7ff3200cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 --2- 172.21.15.3:0/1514550053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff320155df0 0x7ff3200cc100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff310009840 tx=0x7ff310009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 -- 172.21.15.3:0/1514550053 shutdown_connections 2023-11-04T05:29:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 --2- 172.21.15.3:0/1514550053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff32006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 --2- 172.21.15.3:0/1514550053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff320155df0 0x7ff3200cc100 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 -- 172.21.15.3:0/1514550053 >> 172.21.15.3:0/1514550053 conn(0x7ff320072560 msgr2=0x7ff320099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 -- 172.21.15.3:0/1514550053 shutdown_connections 2023-11-04T05:29:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.121+0000 7ff3264c3700 1 -- 172.21.15.3:0/1514550053 wait complete. 2023-11-04T05:29:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.122+0000 7ff3264c3700 1 Processor -- start 2023-11-04T05:29:25.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 -- start start 2023-11-04T05:29:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff320155df0 0x7ff320154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 0x7ff3200b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff3200b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201561d0 con 0x7ff32015be50 2023-11-04T05:29:25.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320156d50 con 0x7ff320155df0 2023-11-04T05:29:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff3264c3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320155050 con 0x7ff32015c2b0 2023-11-04T05:29:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff31f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 0x7ff3200b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff31f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 0x7ff3200b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53426/0 (socket says 172.21.15.3:53426) 2023-11-04T05:29:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.123+0000 7ff31f7fe700 1 -- 172.21.15.3:0/4245836631 learned_addr learned my addr 172.21.15.3:0/4245836631 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:25.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff324a60700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff3200b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31ffff700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff320155df0 0x7ff320154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31f7fe700 1 -- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 msgr2=0x7ff3200b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31f7fe700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff3200b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31f7fe700 1 -- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff320155df0 msgr2=0x7ff320154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31f7fe700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff320155df0 0x7ff320154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31f7fe700 1 -- 172.21.15.3:0/4245836631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100094a0 con 0x7ff32015be50 2023-11-04T05:29:25.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31f7fe700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 0x7ff3200b3bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c730 tx=0x7ff30800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308004c80 con 0x7ff32015be50 2023-11-04T05:29:25.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308004e00 con 0x7ff32015be50 2023-11-04T05:29:25.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320156420 con 0x7ff32015be50 2023-11-04T05:29:25.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308007310 con 0x7ff32015be50 2023-11-04T05:29:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.124+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3201569e0 con 0x7ff32015be50 2023-11-04T05:29:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.125+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32006b960 con 0x7ff32015be50 2023-11-04T05:29:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.126+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff308007470 con 0x7ff32015be50 2023-11-04T05:29:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.126+0000 7ff31cff9700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff30006af00 0x7ff30006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.126+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff3080893e0 con 0x7ff32015be50 2023-11-04T05:29:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.126+0000 7ff31ffff700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff30006af00 0x7ff30006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.127+0000 7ff31ffff700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff30006af00 0x7ff30006d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff3100090b0 tx=0x7ff310008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.131+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff308054160 con 0x7ff32015be50 2023-11-04T05:29:25.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.331+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3200b0f20 con 0x7ff32015be50 2023-11-04T05:29:25.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.331+0000 7ff31cff9700 1 -- 172.21.15.3:0/4245836631 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3200b0f20 con 0x7ff32015be50 2023-11-04T05:29:25.333 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:25.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff30006af00 msgr2=0x7ff30006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff30006af00 0x7ff30006d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff3100090b0 tx=0x7ff310008030 comp rx=0 tx=0).stop 2023-11-04T05:29:25.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 msgr2=0x7ff3200b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 0x7ff3200b3bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c730 tx=0x7ff30800ca40 comp rx=0 tx=0).stop 2023-11-04T05:29:25.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 shutdown_connections 2023-11-04T05:29:25.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff30006af00 0x7ff30006d3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff320155df0 0x7ff320154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff32015c2b0 0x7ff3200b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 --2- 172.21.15.3:0/4245836631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff32015be50 0x7ff3200b3bc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.333+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 >> 172.21.15.3:0/4245836631 conn(0x7ff320072560 msgr2=0x7ff32015d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:25.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.334+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 shutdown_connections 2023-11-04T05:29:25.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.334+0000 7ff3264c3700 1 -- 172.21.15.3:0/4245836631 wait complete. 2023-11-04T05:29:25.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:25.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.475+0000 7f1c2a33f700 1 Processor -- start 2023-11-04T05:29:25.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.475+0000 7f1c2a33f700 1 -- start start 2023-11-04T05:29:25.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.475+0000 7f1c2a33f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c240ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.475+0000 7f1c2a33f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c240cd4b0 0x7f1c2406ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c2a33f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c2406b060 0x7f1c240aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c2a33f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240ce0a0 con 0x7f1c2406b060 2023-11-04T05:29:25.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c2a33f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2406b4f0 con 0x7f1c240cd4b0 2023-11-04T05:29:25.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c2a33f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c240caca0 con 0x7f1c241611d0 2023-11-04T05:29:25.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c2406b060 0x7f1c240aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c2406b060 0x7f1c240aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53436/0 (socket says 172.21.15.3:53436) 2023-11-04T05:29:25.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 -- 172.21.15.3:0/3037552575 learned_addr learned my addr 172.21.15.3:0/3037552575 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:25.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c23fff700 1 --2- 172.21.15.3:0/3037552575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c240ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c237fe700 1 --2- 172.21.15.3:0/3037552575 >> v2:172.21.15.188:3300/0 conn(0x7f1c240cd4b0 0x7f1c2406ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 -- 172.21.15.3:0/3037552575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 msgr2=0x7f1c240ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:25.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 --2- 172.21.15.3:0/3037552575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c240ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 -- 172.21.15.3:0/3037552575 >> v2:172.21.15.188:3300/0 conn(0x7f1c240cd4b0 msgr2=0x7f1c2406ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:25.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 --2- 172.21.15.3:0/3037552575 >> v2:172.21.15.188:3300/0 conn(0x7f1c240cd4b0 0x7f1c2406ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 -- 172.21.15.3:0/3037552575 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240cfe40 con 0x7f1c2406b060 2023-11-04T05:29:25.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c237fe700 1 --2- 172.21.15.3:0/3037552575 >> v2:172.21.15.188:3300/0 conn(0x7f1c240cd4b0 0x7f1c2406ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:25.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c288dc700 1 --2- 172.21.15.3:0/3037552575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240aff20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c240730e0 tx=0x7f1c1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec98579a5ad7c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c22ffd700 1 -- 172.21.15.3:0/3037552575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00e070 con 0x7f1c2406b060 2023-11-04T05:29:25.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c22ffd700 1 -- 172.21.15.3:0/3037552575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00b9c0 con 0x7f1c2406b060 2023-11-04T05:29:25.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.476+0000 7f1c22ffd700 1 -- 172.21.15.3:0/3037552575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00bd70 con 0x7f1c2406b060 2023-11-04T05:29:25.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 -- 172.21.15.3:0/3037552575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 msgr2=0x7f1c240aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/3037552575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240aff20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c240730e0 tx=0x7f1c1c00b330 comp rx=0 tx=0).stop 2023-11-04T05:29:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 -- 172.21.15.3:0/3037552575 shutdown_connections 2023-11-04T05:29:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/3037552575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240aff20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/3037552575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c240ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/3037552575 >> v2:172.21.15.188:3300/0 conn(0x7f1c240cd4b0 0x7f1c2406ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 -- 172.21.15.3:0/3037552575 >> 172.21.15.3:0/3037552575 conn(0x7f1c24072560 msgr2=0x7f1c24099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 -- 172.21.15.3:0/3037552575 shutdown_connections 2023-11-04T05:29:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.477+0000 7f1c2a33f700 1 -- 172.21.15.3:0/3037552575 wait complete. 2023-11-04T05:29:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.478+0000 7f1c2a33f700 1 Processor -- start 2023-11-04T05:29:25.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.478+0000 7f1c2a33f700 1 -- start start 2023-11-04T05:29:25.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c2a33f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c2a33f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c240cd4b0 0x7f1c241f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c2a33f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c241f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c2a33f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c240ca5f0 con 0x7f1c2406b060 2023-11-04T05:29:25.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c2a33f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24158410 con 0x7f1c240cd4b0 2023-11-04T05:29:25.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c2a33f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c240c7e00 con 0x7f1c241611d0 2023-11-04T05:29:25.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53438/0 (socket says 172.21.15.3:53438) 2023-11-04T05:29:25.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 -- 172.21.15.3:0/2351079150 learned_addr learned my addr 172.21.15.3:0/2351079150 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:25.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c237fe700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c240cd4b0 0x7f1c241f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c288dc700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c241f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 -- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 msgr2=0x7f1c241f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c241f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 -- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c240cd4b0 msgr2=0x7f1c241f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.479+0000 7f1c23fff700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c240cd4b0 0x7f1c241f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c23fff700 1 -- 172.21.15.3:0/2351079150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c009b30 con 0x7f1c2406b060 2023-11-04T05:29:25.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c288dc700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c241f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:25.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c237fe700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c240cd4b0 0x7f1c241f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:25.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c23fff700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240ca0b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400d830 tx=0x7f1c1400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c140125f0 con 0x7f1c2406b060 2023-11-04T05:29:25.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c14012750 con 0x7f1c2406b060 2023-11-04T05:29:25.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c240c6d60 con 0x7f1c2406b060 2023-11-04T05:29:25.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14012b00 con 0x7f1c2406b060 2023-11-04T05:29:25.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.480+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c240c7b20 con 0x7f1c2406b060 2023-11-04T05:29:25.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.481+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c240c6580 con 0x7f1c2406b060 2023-11-04T05:29:25.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.481+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c14012c60 con 0x7f1c2406b060 2023-11-04T05:29:25.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.482+0000 7f1c20ff9700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c1006af00 0x7f1c1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.482+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1c14089ce0 con 0x7f1c2406b060 2023-11-04T05:29:25.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.482+0000 7f1c237fe700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c1006af00 0x7f1c1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.482+0000 7f1c237fe700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c1006af00 0x7f1c1006d3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c2406fbe0 tx=0x7f1c18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.486+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1c14054a60 con 0x7f1c2406b060 2023-11-04T05:29:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.687+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1c240c7040 con 0x7f1c2406b060 2023-11-04T05:29:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.688+0000 7f1c20ff9700 1 -- 172.21.15.3:0/2351079150 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1c14058080 con 0x7f1c2406b060 2023-11-04T05:29:25.689 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:25.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c1006af00 msgr2=0x7f1c1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c1006af00 0x7f1c1006d3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c2406fbe0 tx=0x7f1c18008040 comp rx=0 tx=0).stop 2023-11-04T05:29:25.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 msgr2=0x7f1c240ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240ca0b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400d830 tx=0x7f1c1400dbf0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 shutdown_connections 2023-11-04T05:29:25.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c1006af00 0x7f1c1006d3c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c241611d0 0x7f1c241f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406b060 0x7f1c240ca0b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 --2- 172.21.15.3:0/2351079150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c240cd4b0 0x7f1c241f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 >> 172.21.15.3:0/2351079150 conn(0x7f1c24072560 msgr2=0x7f1c2406ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 shutdown_connections 2023-11-04T05:29:25.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.690+0000 7f1c2a33f700 1 -- 172.21.15.3:0/2351079150 wait complete. 2023-11-04T05:29:25.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T05:29:25.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.828+0000 7f4c309fe700 1 Processor -- start 2023-11-04T05:29:25.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.828+0000 7f4c309fe700 1 -- start start 2023-11-04T05:29:25.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c309fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15be70 0x7f4c2c0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c309fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c2c0cc7a0 0x7f4c2c0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c309fe700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c0cd0a0 con 0x7f4c2c0b4a40 2023-11-04T05:29:25.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c309fe700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c0d0240 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c309fe700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c0ceac0 con 0x7f4c2c15be70 2023-11-04T05:29:25.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c2c0cc7a0 0x7f4c2c0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c2c0cc7a0 0x7f4c2c0ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53450/0 (socket says 172.21.15.3:53450) 2023-11-04T05:29:25.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 -- 172.21.15.3:0/4215519663 learned_addr learned my addr 172.21.15.3:0/4215519663 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:25.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2a59c700 1 --2- 172.21.15.3:0/4215519663 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15be70 0x7f4c2c0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 -- 172.21.15.3:0/4215519663 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15be70 msgr2=0x7f4c2c0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:25.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 --2- 172.21.15.3:0/4215519663 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15be70 0x7f4c2c0ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 -- 172.21.15.3:0/4215519663 >> v1:172.21.15.188:6789/0 conn(0x7f4c2c0b4a40 legacy=0x7f4c2c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 -- 172.21.15.3:0/4215519663 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0ccdc0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2a59c700 1 --2- 172.21.15.3:0/4215519663 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15be70 0x7f4c2c0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.830+0000 7f4c2ad9d700 1 --2- 172.21.15.3:0/4215519663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c0ce580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0cefe0 tx=0x7f4c2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed39c473223d67b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c2959a700 1 -- 172.21.15.3:0/4215519663 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400e070 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c2959a700 1 -- 172.21.15.3:0/4215519663 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400b9c0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c2959a700 1 -- 172.21.15.3:0/4215519663 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400bd70 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 -- 172.21.15.3:0/4215519663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 msgr2=0x7f4c2c0ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 --2- 172.21.15.3:0/4215519663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c0ce580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0cefe0 tx=0x7f4c2400b330 comp rx=0 tx=0).stop 2023-11-04T05:29:25.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 -- 172.21.15.3:0/4215519663 shutdown_connections 2023-11-04T05:29:25.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 --2- 172.21.15.3:0/4215519663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c0ce580 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 --2- 172.21.15.3:0/4215519663 >> v2:172.21.15.188:3300/0 conn(0x7f4c2c15be70 0x7f4c2c0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 -- 172.21.15.3:0/4215519663 >> 172.21.15.3:0/4215519663 conn(0x7f4c2c06f0a0 msgr2=0x7f4c2c06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 -- 172.21.15.3:0/4215519663 shutdown_connections 2023-11-04T05:29:25.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.831+0000 7f4c309fe700 1 -- 172.21.15.3:0/4215519663 wait complete. 2023-11-04T05:29:25.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.833+0000 7f4c309fe700 1 Processor -- start 2023-11-04T05:29:25.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.833+0000 7f4c309fe700 1 -- start start 2023-11-04T05:29:25.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c309fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b4a40 0x7f4c2c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c309fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c309fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c15be70 0x7f4c2c0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c309fe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1566b0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c309fe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c06cf10 con 0x7f4c2c15be70 2023-11-04T05:29:25.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c309fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1f4980 con 0x7f4c2c0b4a40 2023-11-04T05:29:25.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c29d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c29d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53454/0 (socket says 172.21.15.3:53454) 2023-11-04T05:29:25.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c29d9b700 1 -- 172.21.15.3:0/339713879 learned_addr learned my addr 172.21.15.3:0/339713879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:25.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c2a59c700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b4a40 0x7f4c2c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c2ad9d700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c15be70 0x7f4c2c0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c29d9b700 1 -- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b4a40 msgr2=0x7f4c2c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c29d9b700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b4a40 0x7f4c2c06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.834+0000 7f4c29d9b700 1 -- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c15be70 msgr2=0x7f4c2c0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c29d9b700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c15be70 0x7f4c2c0c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c29d9b700 1 -- 172.21.15.3:0/339713879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c24009b30 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c2a59c700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b4a40 0x7f4c2c06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c29d9b700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c157b20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00c4e0 tx=0x7f4c1c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0052d0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c005430 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c06bc30 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0038c0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.835+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c06c4d0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.836+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c06c690 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.836+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c1c003a20 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.837+0000 7f4c1affd700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c0c06af00 0x7f4c0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.837+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4c1c088de0 con 0x7f4c2c0cc7a0 2023-11-04T05:29:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.837+0000 7f4c2a59c700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c0c06af00 0x7f4c0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.838+0000 7f4c2a59c700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c0c06af00 0x7f4c0c06d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009960 tx=0x7f4c20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:25.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:25.841+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4c1c053b60 con 0x7f4c2c0cc7a0 2023-11-04T05:29:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.062+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c2c06c050 con 0x7f4c2c0cc7a0 2023-11-04T05:29:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.063+0000 7f4c1affd700 1 -- 172.21.15.3:0/339713879 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4c2c06c050 con 0x7f4c2c0cc7a0 2023-11-04T05:29:26.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.064+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c0c06af00 msgr2=0x7f4c0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.064+0000 7f4c309fe700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c0c06af00 0x7f4c0c06d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009960 tx=0x7f4c20008040 comp rx=0 tx=0).stop 2023-11-04T05:29:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.064+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 msgr2=0x7f4c2c157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.064+0000 7f4c309fe700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c157b20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00c4e0 tx=0x7f4c1c00c8a0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 shutdown_connections 2023-11-04T05:29:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4c0c06af00 0x7f4c0c06d3c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c15be70 0x7f4c2c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b4a40 0x7f4c2c06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 --2- 172.21.15.3:0/339713879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2c0cc7a0 0x7f4c2c157b20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 >> 172.21.15.3:0/339713879 conn(0x7f4c2c06f0a0 msgr2=0x7f4c2c15ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:26.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 shutdown_connections 2023-11-04T05:29:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.065+0000 7f4c309fe700 1 -- 172.21.15.3:0/339713879 wait complete. 2023-11-04T05:29:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T05:29:26.078 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-04T05:29:26.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:26.079 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-04T05:29:26.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T05:29:26.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:26.123 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-04T05:29:26.181 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-04T05:29:26.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:29:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.307+0000 7feae90a4700 1 Processor -- start 2023-11-04T05:29:26.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.307+0000 7feae90a4700 1 -- start start 2023-11-04T05:29:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae90a4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feae415a7e0 0x7feae40cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae90a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feae40b2c00 0x7feae40ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae90a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae40cecf0 0x7feae40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae90a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40cba20 con 0x7feae40b2c00 2023-11-04T05:29:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae90a4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40cf6a0 con 0x7feae415a7e0 2023-11-04T05:29:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae90a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40ce8e0 con 0x7feae40cecf0 2023-11-04T05:29:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feae40b2c00 0x7feae40ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feae40b2c00 0x7feae40ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53460/0 (socket says 172.21.15.3:53460) 2023-11-04T05:29:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 -- 172.21.15.3:0/4238907767 learned_addr learned my addr 172.21.15.3:0/4238907767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae2d9d700 1 --2- 172.21.15.3:0/4238907767 >> v2:172.21.15.188:3300/0 conn(0x7feae415a7e0 0x7feae40cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 -- 172.21.15.3:0/4238907767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae40cecf0 msgr2=0x7feae40cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:26.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 --2- 172.21.15.3:0/4238907767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae40cecf0 0x7feae40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 -- 172.21.15.3:0/4238907767 >> v2:172.21.15.188:3300/0 conn(0x7feae415a7e0 msgr2=0x7feae40cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:26.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 --2- 172.21.15.3:0/4238907767 >> v2:172.21.15.188:3300/0 conn(0x7feae415a7e0 0x7feae40cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 -- 172.21.15.3:0/4238907767 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40cbc70 con 0x7feae40b2c00 2023-11-04T05:29:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae259c700 1 --2- 172.21.15.3:0/4238907767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40ce3a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feae40729f0 tx=0x7fead8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d201fe53ba21f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae1d9b700 1 -- 172.21.15.3:0/4238907767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8005e30 con 0x7feae40b2c00 2023-11-04T05:29:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae1d9b700 1 -- 172.21.15.3:0/4238907767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead8004030 con 0x7feae40b2c00 2023-11-04T05:29:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.308+0000 7feae1d9b700 1 -- 172.21.15.3:0/4238907767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80043e0 con 0x7feae40b2c00 2023-11-04T05:29:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 -- 172.21.15.3:0/4238907767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 msgr2=0x7feae40ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 --2- 172.21.15.3:0/4238907767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40ce3a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feae40729f0 tx=0x7fead8009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 -- 172.21.15.3:0/4238907767 shutdown_connections 2023-11-04T05:29:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 --2- 172.21.15.3:0/4238907767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae40cecf0 0x7feae40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 --2- 172.21.15.3:0/4238907767 >> v2:172.21.15.188:3300/0 conn(0x7feae415a7e0 0x7feae40cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 --2- 172.21.15.3:0/4238907767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40ce3a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 -- 172.21.15.3:0/4238907767 >> 172.21.15.3:0/4238907767 conn(0x7feae40cfd50 msgr2=0x7feae40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 -- 172.21.15.3:0/4238907767 shutdown_connections 2023-11-04T05:29:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.309+0000 7feae90a4700 1 -- 172.21.15.3:0/4238907767 wait complete. 2023-11-04T05:29:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.310+0000 7feae90a4700 1 Processor -- start 2023-11-04T05:29:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 -- start start 2023-11-04T05:29:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae40cecf0 0x7feae41e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae415a7e0 0x7feae40b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4156480 con 0x7feae40b2c00 2023-11-04T05:29:26.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40b3db0 con 0x7feae40cecf0 2023-11-04T05:29:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae90a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40b4090 con 0x7feae415a7e0 2023-11-04T05:29:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53466/0 (socket says 172.21.15.3:53466) 2023-11-04T05:29:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.311+0000 7feae2d9d700 1 -- 172.21.15.3:0/1562090983 learned_addr learned my addr 172.21.15.3:0/1562090983 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae259c700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae40cecf0 0x7feae41e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae359e700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae415a7e0 0x7feae40b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae2d9d700 1 -- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae415a7e0 msgr2=0x7feae40b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae2d9d700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae415a7e0 0x7feae40b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae2d9d700 1 -- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae40cecf0 msgr2=0x7feae41e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae2d9d700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae40cecf0 0x7feae41e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae2d9d700 1 -- 172.21.15.3:0/1562090983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8009500 con 0x7feae40b2c00 2023-11-04T05:29:26.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae2d9d700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40b4e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fead400c7f0 tx=0x7fead400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:26.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4011800 con 0x7feae40b2c00 2023-11-04T05:29:26.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead4011960 con 0x7feae40b2c00 2023-11-04T05:29:26.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae40b4370 con 0x7feae40b2c00 2023-11-04T05:29:26.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4011cf0 con 0x7feae40b2c00 2023-11-04T05:29:26.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.312+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae40b4930 con 0x7feae40b2c00 2023-11-04T05:29:26.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.313+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4074df0 con 0x7feae40b2c00 2023-11-04T05:29:26.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.314+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fead4018360 con 0x7feae40b2c00 2023-11-04T05:29:26.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.314+0000 7feacb7fe700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fead006af00 0x7fead006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.314+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fead40898b0 con 0x7feae40b2c00 2023-11-04T05:29:26.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.314+0000 7feae259c700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fead006af00 0x7fead006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.315+0000 7feae259c700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fead006af00 0x7fead006d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feae40729f0 tx=0x7fead8008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:26.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.319+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fead40545a0 con 0x7feae40b2c00 2023-11-04T05:29:26.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.563+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feae40cddf0 con 0x7feae40b2c00 2023-11-04T05:29:26.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.564+0000 7feacb7fe700 1 -- 172.21.15.3:0/1562090983 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7fead4057bc0 con 0x7feae40b2c00 2023-11-04T05:29:26.565 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:26.566 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:29:26.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fead006af00 msgr2=0x7fead006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fead006af00 0x7fead006d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feae40729f0 tx=0x7fead8008070 comp rx=0 tx=0).stop 2023-11-04T05:29:26.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 msgr2=0x7feae40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40b4e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fead400c7f0 tx=0x7fead400cbb0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 shutdown_connections 2023-11-04T05:29:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fead006af00 0x7fead006d3c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feae415a7e0 0x7feae40b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feae40b2c00 0x7feae40b4e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 --2- 172.21.15.3:0/1562090983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae40cecf0 0x7feae41e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 >> 172.21.15.3:0/1562090983 conn(0x7feae40cfd50 msgr2=0x7feae406b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 shutdown_connections 2023-11-04T05:29:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.566+0000 7feae90a4700 1 -- 172.21.15.3:0/1562090983 wait complete. 2023-11-04T05:29:26.570 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:29:26.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:26.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.704+0000 7f1ebc746700 1 Processor -- start 2023-11-04T05:29:26.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.704+0000 7f1ebc746700 1 -- start start 2023-11-04T05:29:26.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1ebc746700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 0x7f1eb40728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1ebc746700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1eb406af00 0x7f1eb40d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1ebc746700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb4072df0 con 0x7f1eb40cc7a0 2023-11-04T05:29:26.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1ebc746700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40cf050 con 0x7f1eb406af00 2023-11-04T05:29:26.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1ebc746700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40cda00 con 0x7f1eb406ab20 2023-11-04T05:29:26.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1eb9ce1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1eb40cc7a0 0x7f1eb40cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58822/0 (socket says 172.21.15.3:58822) 2023-11-04T05:29:26.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1eb9ce1700 1 -- 172.21.15.3:0/861253209 learned_addr learned my addr 172.21.15.3:0/861253209 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:26.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1ebace3700 1 --2- 172.21.15.3:0/861253209 >> v2:172.21.15.188:3300/0 conn(0x7f1eb406af00 0x7f1eb40d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284489740 0 0) 0x7f1eb4072df0 con 0x7f1eb40cc7a0 2023-11-04T05:29:26.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 0x7f1eb40728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.705+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea00035b0 con 0x7f1eb40cc7a0 2023-11-04T05:29:26.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965687061 0 0) 0x7f1ea00035b0 con 0x7f1eb40cc7a0 2023-11-04T05:29:26.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 msgr2=0x7f1eb40728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:26.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 0x7f1eb40728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 >> v2:172.21.15.188:3300/0 conn(0x7f1eb406af00 msgr2=0x7f1eb40d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:26.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 --2- 172.21.15.3:0/861253209 >> v2:172.21.15.188:3300/0 conn(0x7f1eb406af00 0x7f1eb40d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40cdce0 con 0x7f1eb40cc7a0 2023-11-04T05:29:26.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f1eb00017f0 con 0x7f1eb40cc7a0 2023-11-04T05:29:26.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 >> v1:172.21.15.3:6789/0 conn(0x7f1eb40cc7a0 legacy=0x7f1eb40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1ea0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 0x7f1ea0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ea0004d90 0x7f1ea0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40cf050 con 0x7f1ea0003a00 2023-11-04T05:29:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40cda00 con 0x7f1ea0004d90 2023-11-04T05:29:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea000ba10 con 0x7f1eb406ab20 2023-11-04T05:29:26.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1ebace3700 1 --2- 172.21.15.3:0/861253209 >> v2:172.21.15.188:3300/0 conn(0x7f1eb406af00 0x7f1eb40d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1ea0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb9ce1700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 0x7f1ea0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1ebace3700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ea0004d90 0x7f1ea0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 -- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 msgr2=0x7f1ea0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb406ab20 0x7f1ea0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 -- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ea0004d90 msgr2=0x7f1ea0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ea0004d90 0x7f1ea0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 -- 172.21.15.3:0/861253209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00035b0 con 0x7f1ea0003a00 2023-11-04T05:29:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1ea0003e20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009b90 tx=0x7f1ea0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98085ee63b28de9 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800e6c0 con 0x7f1ea0003a00 2023-11-04T05:29:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea800e820 con 0x7f1ea0003a00 2023-11-04T05:29:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.706+0000 7f1eb94e0700 1 -- 172.21.15.3:0/861253209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800eb90 con 0x7f1ea0003a00 2023-11-04T05:29:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebc746700 1 -- 172.21.15.3:0/861253209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 msgr2=0x7f1ea0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebc746700 1 --2- 172.21.15.3:0/861253209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1ea0003e20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009b90 tx=0x7f1ea0005880 comp rx=0 tx=0).stop 2023-11-04T05:29:26.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebace3700 1 -- 172.21.15.3:0/861253209 reap_dead start 2023-11-04T05:29:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebc746700 1 -- 172.21.15.3:0/861253209 shutdown_connections 2023-11-04T05:29:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebc746700 1 -- 172.21.15.3:0/861253209 >> 172.21.15.3:0/861253209 conn(0x7f1eb406df90 msgr2=0x7f1eb406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebc746700 1 -- 172.21.15.3:0/861253209 shutdown_connections 2023-11-04T05:29:26.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.707+0000 7f1ebc746700 1 -- 172.21.15.3:0/861253209 wait complete. 2023-11-04T05:29:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.708+0000 7f1ebc746700 1 Processor -- start 2023-11-04T05:29:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.708+0000 7f1ebc746700 1 -- start start 2023-11-04T05:29:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebc746700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1eb414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebc746700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb4159f40 0x7f1eb40c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebc746700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb40c6b60 0x7f1eb4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebc746700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb4158b80 con 0x7f1ea0003a00 2023-11-04T05:29:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebc746700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c6740 con 0x7f1eb40c6b60 2023-11-04T05:29:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebc746700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c5b10 con 0x7f1eb4159f40 2023-11-04T05:29:26.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1eb414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1eb414ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53484/0 (socket says 172.21.15.3:53484) 2023-11-04T05:29:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 -- 172.21.15.3:0/3602411110 learned_addr learned my addr 172.21.15.3:0/3602411110 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebace3700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb40c6b60 0x7f1eb4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eb9ce1700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb4159f40 0x7f1eb40c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 -- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb4159f40 msgr2=0x7f1eb40c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb4159f40 0x7f1eb40c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 -- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb40c6b60 msgr2=0x7f1eb4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb40c6b60 0x7f1eb4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eba4e2700 1 -- 172.21.15.3:0/3602411110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb415a3c0 con 0x7f1ea0003a00 2023-11-04T05:29:26.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1eb9ce1700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb4159f40 0x7f1eb40c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:26.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.709+0000 7f1ebace3700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb40c6b60 0x7f1eb4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:26.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.710+0000 7f1eba4e2700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1eb414ccc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80090b0 tx=0x7f1ea80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:26.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.710+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800e6c0 con 0x7f1ea0003a00 2023-11-04T05:29:26.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.710+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8003ad0 con 0x7f1ea0003a00 2023-11-04T05:29:26.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.710+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb415a600 con 0x7f1ea0003a00 2023-11-04T05:29:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.710+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8003e80 con 0x7f1ea0003a00 2023-11-04T05:29:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.710+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb40c6080 con 0x7f1ea0003a00 2023-11-04T05:29:26.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.711+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb4074df0 con 0x7f1ea0003a00 2023-11-04T05:29:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.711+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ea8013470 con 0x7f1ea0003a00 2023-11-04T05:29:26.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.712+0000 7f1ea6ffd700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1e9806af00 0x7f1e9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.712+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1ea808b410 con 0x7f1ea0003a00 2023-11-04T05:29:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.712+0000 7f1eb9ce1700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1e9806af00 0x7f1e9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.712+0000 7f1eb9ce1700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1e9806af00 0x7f1e9806d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0000c00 tx=0x7f1eb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.716+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1ea80562b0 con 0x7f1ea0003a00 2023-11-04T05:29:26.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.919+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eb40cd020 con 0x7f1ea0003a00 2023-11-04T05:29:26.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.921+0000 7f1ea6ffd700 1 -- 172.21.15.3:0/3602411110 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7f1ea80598d0 con 0x7f1ea0003a00 2023-11-04T05:29:26.922 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:26.922 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:26.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1e9806af00 msgr2=0x7f1e9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1e9806af00 0x7f1e9806d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0000c00 tx=0x7f1eb0009040 comp rx=0 tx=0).stop 2023-11-04T05:29:26.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 msgr2=0x7f1eb414ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:26.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1eb414ccc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80090b0 tx=0x7f1ea80047c0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 shutdown_connections 2023-11-04T05:29:26.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1e9806af00 0x7f1e9806d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb40c6b60 0x7f1eb4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea0003a00 0x7f1eb414ccc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 --2- 172.21.15.3:0/3602411110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb4159f40 0x7f1eb40c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 >> 172.21.15.3:0/3602411110 conn(0x7f1eb406df90 msgr2=0x7f1eb40d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 shutdown_connections 2023-11-04T05:29:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:26.923+0000 7f1ebc746700 1 -- 172.21.15.3:0/3602411110 wait complete. 2023-11-04T05:29:26.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:28:18.812474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:28:19.827692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-04T05:28:23.853174+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-04T05:29:26.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:27.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.065+0000 7f19c903f700 1 Processor -- start 2023-11-04T05:29:27.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.065+0000 7f19c903f700 1 -- start start 2023-11-04T05:29:27.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c903f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19c40d0d70 0x7f19c40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c903f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19c40b3650 0x7f19c415f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c903f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c4070a90 con 0x7f19c415ba10 2023-11-04T05:29:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c903f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40b3ac0 con 0x7f19c40d0d70 2023-11-04T05:29:27.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c903f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40718f0 con 0x7f19c40b3650 2023-11-04T05:29:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19c40d0d70 0x7f19c40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c2d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f19c415ba10 0x7f19c40c51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58840/0 (socket says 172.21.15.3:58840) 2023-11-04T05:29:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.066+0000 7f19c2d9d700 1 -- 172.21.15.3:0/1315232214 learned_addr learned my addr 172.21.15.3:0/1315232214 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c359e700 1 --2- 172.21.15.3:0/1315232214 >> v2:172.21.15.188:3300/0 conn(0x7f19c40b3650 0x7f19c415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c259c700 1 -- 172.21.15.3:0/1315232214 >> v2:172.21.15.188:3300/0 conn(0x7f19c40b3650 msgr2=0x7f19c415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:27.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c259c700 1 --2- 172.21.15.3:0/1315232214 >> v2:172.21.15.188:3300/0 conn(0x7f19c40b3650 0x7f19c415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c259c700 1 -- 172.21.15.3:0/1315232214 >> v1:172.21.15.3:6789/0 conn(0x7f19c415ba10 legacy=0x7f19c40c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c259c700 1 -- 172.21.15.3:0/1315232214 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4070ce0 con 0x7f19c40d0d70 2023-11-04T05:29:27.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c1d9b700 1 -- 172.21.15.3:0/1315232214 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789586124 0 0) 0x7f19c4070a90 con 0x7f19c415ba10 2023-11-04T05:29:27.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c259c700 1 --2- 172.21.15.3:0/1315232214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d0d70 0x7f19c40b3110 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b4260 tx=0x7f19b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1d795c7fa1d6f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c1d9b700 1 -- 172.21.15.3:0/1315232214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4011830 con 0x7f19c40d0d70 2023-11-04T05:29:27.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c1d9b700 1 -- 172.21.15.3:0/1315232214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b4011990 con 0x7f19c40d0d70 2023-11-04T05:29:27.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c1d9b700 1 -- 172.21.15.3:0/1315232214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4011d20 con 0x7f19c40d0d70 2023-11-04T05:29:27.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c903f700 1 -- 172.21.15.3:0/1315232214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d0d70 msgr2=0x7f19c40b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.067+0000 7f19c903f700 1 --2- 172.21.15.3:0/1315232214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d0d70 0x7f19c40b3110 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b4260 tx=0x7f19b4009b20 comp rx=0 tx=0).stop 2023-11-04T05:29:27.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.068+0000 7f19c903f700 1 -- 172.21.15.3:0/1315232214 shutdown_connections 2023-11-04T05:29:27.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.068+0000 7f19c903f700 1 --2- 172.21.15.3:0/1315232214 >> v2:172.21.15.188:3300/0 conn(0x7f19c40b3650 0x7f19c415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.068+0000 7f19c903f700 1 --2- 172.21.15.3:0/1315232214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d0d70 0x7f19c40b3110 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.068+0000 7f19c903f700 1 -- 172.21.15.3:0/1315232214 >> 172.21.15.3:0/1315232214 conn(0x7f19c40ce110 msgr2=0x7f19c40ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:27.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.068+0000 7f19c903f700 1 -- 172.21.15.3:0/1315232214 shutdown_connections 2023-11-04T05:29:27.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.068+0000 7f19c903f700 1 -- 172.21.15.3:0/1315232214 wait complete. 2023-11-04T05:29:27.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.069+0000 7f19c903f700 1 Processor -- start 2023-11-04T05:29:27.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.069+0000 7f19c903f700 1 -- start start 2023-11-04T05:29:27.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c903f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c40b3650 0x7f19c41594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c903f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c40d0d70 0x7f19c4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c903f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 0x7f19c41e4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c903f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c41f13d0 con 0x7f19c415ba10 2023-11-04T05:29:27.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c903f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4147340 con 0x7f19c40d0d70 2023-11-04T05:29:27.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c903f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4147620 con 0x7f19c40b3650 2023-11-04T05:29:27.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 0x7f19c41e4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 0x7f19c41e4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53500/0 (socket says 172.21.15.3:53500) 2023-11-04T05:29:27.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 -- 172.21.15.3:0/3240764034 learned_addr learned my addr 172.21.15.3:0/3240764034 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:27.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c259c700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c40d0d70 0x7f19c4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c2d9d700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c40b3650 0x7f19c41594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 -- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c40b3650 msgr2=0x7f19c41594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c40b3650 0x7f19c41594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 -- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c40d0d70 msgr2=0x7f19c4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c40d0d70 0x7f19c4159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 -- 172.21.15.3:0/3240764034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b40094a0 con 0x7f19c415ba10 2023-11-04T05:29:27.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.070+0000 7f19c359e700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 0x7f19c41e4560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f19bc007e60 tx=0x7f19bc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.071+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00ede0 con 0x7f19c415ba10 2023-11-04T05:29:27.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.071+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc014420 con 0x7f19c415ba10 2023-11-04T05:29:27.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.071+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c4147900 con 0x7f19c415ba10 2023-11-04T05:29:27.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.071+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc0147d0 con 0x7f19c415ba10 2023-11-04T05:29:27.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.071+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c4147ec0 con 0x7f19c415ba10 2023-11-04T05:29:27.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.072+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c4074df0 con 0x7f19c415ba10 2023-11-04T05:29:27.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.072+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19bc014930 con 0x7f19c415ba10 2023-11-04T05:29:27.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.072+0000 7f19ab7fe700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f19b006af00 0x7f19b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.073+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f19bc08a100 con 0x7f19c415ba10 2023-11-04T05:29:27.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.073+0000 7f19c2d9d700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f19b006af00 0x7f19b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.073+0000 7f19c2d9d700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f19b006af00 0x7f19b006d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19b8000f60 tx=0x7f19b8007be0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.077+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f19bc054df0 con 0x7f19c415ba10 2023-11-04T05:29:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.278+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19c40cee70 con 0x7f19c415ba10 2023-11-04T05:29:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.278+0000 7f19ab7fe700 1 -- 172.21.15.3:0/3240764034 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19bc058410 con 0x7f19c415ba10 2023-11-04T05:29:27.280 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f19b006af00 msgr2=0x7f19b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f19b006af00 0x7f19b006d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19b8000f60 tx=0x7f19b8007be0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 msgr2=0x7f19c41e4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 0x7f19c41e4560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f19bc007e60 tx=0x7f19bc00e3c0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 shutdown_connections 2023-11-04T05:29:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f19b006af00 0x7f19b006d3c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c415ba10 0x7f19c41e4560 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c40b3650 0x7f19c41594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 --2- 172.21.15.3:0/3240764034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c40d0d70 0x7f19c4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 >> 172.21.15.3:0/3240764034 conn(0x7f19c40ce110 msgr2=0x7f19c415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 shutdown_connections 2023-11-04T05:29:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.280+0000 7f19c903f700 1 -- 172.21.15.3:0/3240764034 wait complete. 2023-11-04T05:29:27.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T05:29:27.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.420+0000 7f6c32af3700 1 Processor -- start 2023-11-04T05:29:27.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 -- start start 2023-11-04T05:29:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6c2c1611d0 0x7f6c2c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c2c0cd4b0 0x7f6c2c06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c06b060 0x7f6c2c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0ce0a0 con 0x7f6c2c0cd4b0 2023-11-04T05:29:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c06b4f0 con 0x7f6c2c1611d0 2023-11-04T05:29:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c32af3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0caca0 con 0x7f6c2c06b060 2023-11-04T05:29:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.421+0000 7f6c2bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c2c0cd4b0 0x7f6c2c06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c2c0cd4b0 0x7f6c2c06ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53506/0 (socket says 172.21.15.3:53506) 2023-11-04T05:29:27.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 -- 172.21.15.3:0/3607188899 learned_addr learned my addr 172.21.15.3:0/3607188899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c3088f700 1 --2- 172.21.15.3:0/3607188899 >> v2:172.21.15.188:3300/0 conn(0x7f6c2c1611d0 0x7f6c2c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c31090700 1 --2- 172.21.15.3:0/3607188899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c06b060 0x7f6c2c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 -- 172.21.15.3:0/3607188899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c06b060 msgr2=0x7f6c2c0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/3607188899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c06b060 0x7f6c2c0aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 -- 172.21.15.3:0/3607188899 >> v2:172.21.15.188:3300/0 conn(0x7f6c2c1611d0 msgr2=0x7f6c2c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/3607188899 >> v2:172.21.15.188:3300/0 conn(0x7f6c2c1611d0 0x7f6c2c0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 -- 172.21.15.3:0/3607188899 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0cfe40 con 0x7f6c2c0cd4b0 2023-11-04T05:29:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c3088f700 1 --2- 172.21.15.3:0/3607188899 >> v2:172.21.15.188:3300/0 conn(0x7f6c2c1611d0 0x7f6c2c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/3607188899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c0cd4b0 0x7f6c2c06ab20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c072aa0 tx=0x7f6c18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bf55aa8825abc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2b7fe700 1 -- 172.21.15.3:0/3607188899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800b870 con 0x7f6c2c0cd4b0 2023-11-04T05:29:27.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2b7fe700 1 -- 172.21.15.3:0/3607188899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1800b9d0 con 0x7f6c2c0cd4b0 2023-11-04T05:29:27.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.422+0000 7f6c2b7fe700 1 -- 172.21.15.3:0/3607188899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800bd60 con 0x7f6c2c0cd4b0 2023-11-04T05:29:27.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 -- 172.21.15.3:0/3607188899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c0cd4b0 msgr2=0x7f6c2c06ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 --2- 172.21.15.3:0/3607188899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c0cd4b0 0x7f6c2c06ab20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c072aa0 tx=0x7f6c18009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 -- 172.21.15.3:0/3607188899 shutdown_connections 2023-11-04T05:29:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 --2- 172.21.15.3:0/3607188899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c06b060 0x7f6c2c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 --2- 172.21.15.3:0/3607188899 >> v2:172.21.15.188:3300/0 conn(0x7f6c2c1611d0 0x7f6c2c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 --2- 172.21.15.3:0/3607188899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c0cd4b0 0x7f6c2c06ab20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 -- 172.21.15.3:0/3607188899 >> 172.21.15.3:0/3607188899 conn(0x7f6c2c072560 msgr2=0x7f6c2c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:27.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 -- 172.21.15.3:0/3607188899 shutdown_connections 2023-11-04T05:29:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.423+0000 7f6c32af3700 1 -- 172.21.15.3:0/3607188899 wait complete. 2023-11-04T05:29:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.424+0000 7f6c32af3700 1 Processor -- start 2023-11-04T05:29:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.424+0000 7f6c32af3700 1 -- start start 2023-11-04T05:29:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 0x7f6c2c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c0cd4b0 0x7f6c2c0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c1611d0 0x7f6c2c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1f2eb0 con 0x7f6c2c06b060 2023-11-04T05:29:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c7720 con 0x7f6c2c1611d0 2023-11-04T05:29:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c6ce0 con 0x7f6c2c0cd4b0 2023-11-04T05:29:27.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 0x7f6c2c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 0x7f6c2c15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53512/0 (socket says 172.21.15.3:53512) 2023-11-04T05:29:27.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 -- 172.21.15.3:0/2227579307 learned_addr learned my addr 172.21.15.3:0/2227579307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:27.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c0cd4b0 0x7f6c2c0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 -- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c0cd4b0 msgr2=0x7f6c2c0c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c31090700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c1611d0 0x7f6c2c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c0cd4b0 0x7f6c2c0c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 -- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c1611d0 msgr2=0x7f6c2c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c1611d0 0x7f6c2c0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 -- 172.21.15.3:0/2227579307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18009550 con 0x7f6c2c06b060 2023-11-04T05:29:27.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c0cd4b0 0x7f6c2c0c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:27.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c3088f700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 0x7f6c2c15c760 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000e810 tx=0x7f6c2000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000c020 con 0x7f6c2c06b060 2023-11-04T05:29:27.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20004250 con 0x7f6c2c06b060 2023-11-04T05:29:27.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c15cca0 con 0x7f6c2c06b060 2023-11-04T05:29:27.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200036a0 con 0x7f6c2c06b060 2023-11-04T05:29:27.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.426+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c0c60d0 con 0x7f6c2c06b060 2023-11-04T05:29:27.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.427+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c074df0 con 0x7f6c2c06b060 2023-11-04T05:29:27.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.428+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c20003800 con 0x7f6c2c06b060 2023-11-04T05:29:27.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.428+0000 7f6c297fa700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6c1006af00 0x7f6c1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.428+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6c20089410 con 0x7f6c2c06b060 2023-11-04T05:29:27.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.428+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6c1006af00 0x7f6c1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.429+0000 7f6c2bfff700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6c1006af00 0x7f6c1006d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0c5b90 tx=0x7f6c18008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.433+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f6c20054100 con 0x7f6c2c06b060 2023-11-04T05:29:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.633+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c2c0d0c80 con 0x7f6c2c06b060 2023-11-04T05:29:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.634+0000 7f6c297fa700 1 -- 172.21.15.3:0/2227579307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c20057720 con 0x7f6c2c06b060 2023-11-04T05:29:27.635 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6c1006af00 msgr2=0x7f6c1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6c1006af00 0x7f6c1006d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0c5b90 tx=0x7f6c18008030 comp rx=0 tx=0).stop 2023-11-04T05:29:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 msgr2=0x7f6c2c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 0x7f6c2c15c760 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000e810 tx=0x7f6c2000ebd0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 shutdown_connections 2023-11-04T05:29:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6c1006af00 0x7f6c1006d3c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c1611d0 0x7f6c2c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c2c06b060 0x7f6c2c15c760 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 --2- 172.21.15.3:0/2227579307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2c0cd4b0 0x7f6c2c0c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 >> 172.21.15.3:0/2227579307 conn(0x7f6c2c072560 msgr2=0x7f6c2c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 shutdown_connections 2023-11-04T05:29:27.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.636+0000 7f6c32af3700 1 -- 172.21.15.3:0/2227579307 wait complete. 2023-11-04T05:29:27.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T05:29:27.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.776+0000 7ff3b296c700 1 Processor -- start 2023-11-04T05:29:27.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.777+0000 7ff3b296c700 1 -- start start 2023-11-04T05:29:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.777+0000 7ff3b296c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff3ac0cc100 0x7ff3ac15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.777+0000 7ff3b296c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac1601a0 0x7ff3ac06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.777+0000 7ff3b296c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac0b0f20 con 0x7ff3ac155df0 2023-11-04T05:29:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.777+0000 7ff3b296c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cc590 con 0x7ff3ac0cc100 2023-11-04T05:29:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.777+0000 7ff3b296c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac15c560 con 0x7ff3ac1601a0 2023-11-04T05:29:27.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff3ac0cc100 0x7ff3ac15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff3ac0cc100 0x7ff3ac15fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53522/0 (socket says 172.21.15.3:53522) 2023-11-04T05:29:27.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 -- 172.21.15.3:0/1389659278 learned_addr learned my addr 172.21.15.3:0/1389659278 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:27.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 -- 172.21.15.3:0/1389659278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac1601a0 msgr2=0x7ff3ac06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:27.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3b0f09700 1 --2- 172.21.15.3:0/1389659278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac1601a0 0x7ff3ac06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 --2- 172.21.15.3:0/1389659278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac1601a0 0x7ff3ac06ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 -- 172.21.15.3:0/1389659278 >> v1:172.21.15.188:6789/0 conn(0x7ff3ac155df0 legacy=0x7ff3ac15be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:27.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 -- 172.21.15.3:0/1389659278 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac15f4a0 con 0x7ff3ac0cc100 2023-11-04T05:29:27.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3ab7fe700 1 --2- 172.21.15.3:0/1389659278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac15fc60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00098f0 tx=0x7ff3a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b84525f2a681294 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3aaffd700 1 -- 172.21.15.3:0/1389659278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004230 con 0x7ff3ac0cc100 2023-11-04T05:29:27.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3aaffd700 1 -- 172.21.15.3:0/1389659278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0004390 con 0x7ff3ac0cc100 2023-11-04T05:29:27.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.778+0000 7ff3aaffd700 1 -- 172.21.15.3:0/1389659278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00036a0 con 0x7ff3ac0cc100 2023-11-04T05:29:27.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 -- 172.21.15.3:0/1389659278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0cc100 msgr2=0x7ff3ac15fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 --2- 172.21.15.3:0/1389659278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac15fc60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00098f0 tx=0x7ff3a0009c00 comp rx=0 tx=0).stop 2023-11-04T05:29:27.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 -- 172.21.15.3:0/1389659278 shutdown_connections 2023-11-04T05:29:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 --2- 172.21.15.3:0/1389659278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac1601a0 0x7ff3ac06ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 --2- 172.21.15.3:0/1389659278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac15fc60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 -- 172.21.15.3:0/1389659278 >> 172.21.15.3:0/1389659278 conn(0x7ff3ac072560 msgr2=0x7ff3ac099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 -- 172.21.15.3:0/1389659278 shutdown_connections 2023-11-04T05:29:27.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.779+0000 7ff3b296c700 1 -- 172.21.15.3:0/1389659278 wait complete. 2023-11-04T05:29:27.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.780+0000 7ff3b296c700 1 Processor -- start 2023-11-04T05:29:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.780+0000 7ff3b296c700 1 -- start start 2023-11-04T05:29:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b296c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac0c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b296c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac155df0 0x7ff3ac14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b296c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 0x7ff3ac157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b296c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c63c0 con 0x7ff3ac0c6850 2023-11-04T05:29:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b296c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac156470 con 0x7ff3ac0cc100 2023-11-04T05:29:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b296c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c57b0 con 0x7ff3ac155df0 2023-11-04T05:29:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 0x7ff3ac157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 0x7ff3ac157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53530/0 (socket says 172.21.15.3:53530) 2023-11-04T05:29:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 -- 172.21.15.3:0/4210947679 learned_addr learned my addr 172.21.15.3:0/4210947679 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3abfff700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac0c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3ab7fe700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac155df0 0x7ff3ac14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 -- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac155df0 msgr2=0x7ff3ac14a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac155df0 0x7ff3ac14a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 -- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac0cc100 msgr2=0x7ff3ac0c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac0c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.781+0000 7ff3b0f09700 1 -- 172.21.15.3:0/4210947679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a0009550 con 0x7ff3ac0c6850 2023-11-04T05:29:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.782+0000 7ff3b0f09700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 0x7ff3ac157f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400cc00 tx=0x7ff3a400a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.782+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400ab30 con 0x7ff3ac0c6850 2023-11-04T05:29:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.782+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400ac90 con 0x7ff3ac0c6850 2023-11-04T05:29:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.782+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac0c5a90 con 0x7ff3ac0c6850 2023-11-04T05:29:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.782+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4003a20 con 0x7ff3ac0c6850 2023-11-04T05:29:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.782+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac0c6050 con 0x7ff3ac0c6850 2023-11-04T05:29:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.783+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac0c6530 con 0x7ff3ac0c6850 2023-11-04T05:29:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.783+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3a400f3f0 con 0x7ff3ac0c6850 2023-11-04T05:29:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.784+0000 7ff3a8ff9700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff39806af00 0x7ff39806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.784+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff3a4089b20 con 0x7ff3ac0c6850 2023-11-04T05:29:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.784+0000 7ff3abfff700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff39806af00 0x7ff39806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.784+0000 7ff3abfff700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff39806af00 0x7ff39806d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac14a8e0 tx=0x7ff39c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.788+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff3a40548a0 con 0x7ff3ac0c6850 2023-11-04T05:29:27.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.992+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3ac06b960 con 0x7ff3ac0c6850 2023-11-04T05:29:27.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.993+0000 7ff3a8ff9700 1 -- 172.21.15.3:0/4210947679 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3a401d070 con 0x7ff3ac0c6850 2023-11-04T05:29:27.994 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T05:29:27.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff39806af00 msgr2=0x7ff39806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff39806af00 0x7ff39806d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac14a8e0 tx=0x7ff39c009040 comp rx=0 tx=0).stop 2023-11-04T05:29:27.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 msgr2=0x7ff3ac157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:27.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 0x7ff3ac157f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400cc00 tx=0x7ff3a400a1f0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 shutdown_connections 2023-11-04T05:29:27.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff39806af00 0x7ff39806d3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac0cc100 0x7ff3ac0c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ac0c6850 0x7ff3ac157f20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 --2- 172.21.15.3:0/4210947679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac155df0 0x7ff3ac14a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:27.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 >> 172.21.15.3:0/4210947679 conn(0x7ff3ac072560 msgr2=0x7ff3ac15bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:27.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 shutdown_connections 2023-11-04T05:29:27.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:27.995+0000 7ff3b296c700 1 -- 172.21.15.3:0/4210947679 wait complete. 2023-11-04T05:29:28.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-04T05:29:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.135+0000 7f1efb669700 1 Processor -- start 2023-11-04T05:29:28.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.136+0000 7f1efb669700 1 -- start start 2023-11-04T05:29:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.136+0000 7f1efb669700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ef406e2f0 0x7f1ef40668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.136+0000 7f1efb669700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 0x7f1ef4066e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.136+0000 7f1efb669700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef40cdce0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.136+0000 7f1efb669700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef40cc590 con 0x7f1ef406e2f0 2023-11-04T05:29:28.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.136+0000 7f1efb669700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef415e1d0 con 0x7f1ef40b2f30 2023-11-04T05:29:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 0x7f1ef4066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef8c04700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ef406e2f0 0x7f1ef40668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 0x7f1ef4066e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:44594/0 (socket says 172.21.15.3:44594) 2023-11-04T05:29:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 -- 172.21.15.3:0/2588857169 learned_addr learned my addr 172.21.15.3:0/2588857169 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:28.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 -- 172.21.15.3:0/2588857169 >> v2:172.21.15.188:3300/0 conn(0x7f1ef406e2f0 msgr2=0x7f1ef40668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 --2- 172.21.15.3:0/2588857169 >> v2:172.21.15.188:3300/0 conn(0x7f1ef406e2f0 0x7f1ef40668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 -- 172.21.15.3:0/2588857169 >> v1:172.21.15.188:6789/0 conn(0x7f1ef415ddf0 legacy=0x7f1ef40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 -- 172.21.15.3:0/2588857169 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40cdfc0 con 0x7f1ef40b2f30 2023-11-04T05:29:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef8c04700 1 --2- 172.21.15.3:0/2588857169 >> v2:172.21.15.188:3300/0 conn(0x7f1ef406e2f0 0x7f1ef40668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.137+0000 7f1ef9c06700 1 --2- 172.21.15.3:0/2588857169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 0x7f1ef4066e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009e50 tx=0x7f1eec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705202226ece00ee server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1ee7fff700 1 -- 172.21.15.3:0/2588857169 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00e070 con 0x7f1ef40b2f30 2023-11-04T05:29:28.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1ee7fff700 1 -- 172.21.15.3:0/2588857169 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec00b910 con 0x7f1ef40b2f30 2023-11-04T05:29:28.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1ee7fff700 1 -- 172.21.15.3:0/2588857169 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00bcc0 con 0x7f1ef40b2f30 2023-11-04T05:29:28.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 -- 172.21.15.3:0/2588857169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 msgr2=0x7f1ef4066e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 --2- 172.21.15.3:0/2588857169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 0x7f1ef4066e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009e50 tx=0x7f1eec00b330 comp rx=0 tx=0).stop 2023-11-04T05:29:28.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 -- 172.21.15.3:0/2588857169 shutdown_connections 2023-11-04T05:29:28.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 --2- 172.21.15.3:0/2588857169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef40b2f30 0x7f1ef4066e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 --2- 172.21.15.3:0/2588857169 >> v2:172.21.15.188:3300/0 conn(0x7f1ef406e2f0 0x7f1ef40668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 -- 172.21.15.3:0/2588857169 >> 172.21.15.3:0/2588857169 conn(0x7f1ef40cc860 msgr2=0x7f1ef40ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:28.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 -- 172.21.15.3:0/2588857169 shutdown_connections 2023-11-04T05:29:28.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.138+0000 7f1efb669700 1 -- 172.21.15.3:0/2588857169 wait complete. 2023-11-04T05:29:28.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.139+0000 7f1efb669700 1 Processor -- start 2023-11-04T05:29:28.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 -- start start 2023-11-04T05:29:28.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef406e2f0 0x7f1ef4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef40b2f30 0x7f1ef414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 0x7f1ef414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef414ea60 con 0x7f1ef415ddf0 2023-11-04T05:29:28.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4150330 con 0x7f1ef40b2f30 2023-11-04T05:29:28.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1efb669700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef41505b0 con 0x7f1ef406e2f0 2023-11-04T05:29:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1ef9c06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 0x7f1ef414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1ef9c06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 0x7f1ef414fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53544/0 (socket says 172.21.15.3:53544) 2023-11-04T05:29:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.140+0000 7f1ef9c06700 1 -- 172.21.15.3:0/3272932522 learned_addr learned my addr 172.21.15.3:0/3272932522 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9405700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef406e2f0 0x7f1ef4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef8c04700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef40b2f30 0x7f1ef414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9c06700 1 -- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef406e2f0 msgr2=0x7f1ef4155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9c06700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef406e2f0 0x7f1ef4155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9c06700 1 -- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef40b2f30 msgr2=0x7f1ef414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9c06700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef40b2f30 0x7f1ef414e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9c06700 1 -- 172.21.15.3:0/3272932522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80094a0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ef9c06700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 0x7f1ef414fdf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1ef415e310 tx=0x7f1eec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0048b0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec0092e0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec009b30 con 0x7f1ef415ddf0 2023-11-04T05:29:28.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec01c680 con 0x7f1ef415ddf0 2023-11-04T05:29:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.141+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef4152e20 con 0x7f1ef415ddf0 2023-11-04T05:29:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.142+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef4150ad0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.143+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1eec01c7e0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.143+0000 7f1ee5ffb700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1ed806af00 0x7f1ed806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.143+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1eec08b590 con 0x7f1ef415ddf0 2023-11-04T05:29:28.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.144+0000 7f1ef9405700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1ed806af00 0x7f1ed806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.144+0000 7f1ef9405700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1ed806af00 0x7f1ed806d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0007fd0 tx=0x7f1ef0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.148+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1ef4150ad0 con 0x7f1ef415ddf0 2023-11-04T05:29:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.346+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1ef4152c80 con 0x7f1ef415ddf0 2023-11-04T05:29:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.347+0000 7f1ee5ffb700 1 -- 172.21.15.3:0/3272932522 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1ef4152c80 con 0x7f1ef415ddf0 2023-11-04T05:29:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T05:29:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1ed806af00 msgr2=0x7f1ed806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1ed806af00 0x7f1ed806d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0007fd0 tx=0x7f1ef0009040 comp rx=0 tx=0).stop 2023-11-04T05:29:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 msgr2=0x7f1ef414fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 0x7f1ef414fdf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1ef415e310 tx=0x7f1eec004ad0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 shutdown_connections 2023-11-04T05:29:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1ed806af00 0x7f1ed806d3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ef415ddf0 0x7f1ef414fdf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef406e2f0 0x7f1ef4155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 --2- 172.21.15.3:0/3272932522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef40b2f30 0x7f1ef414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 >> 172.21.15.3:0/3272932522 conn(0x7f1ef40cc860 msgr2=0x7f1ef406ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 shutdown_connections 2023-11-04T05:29:28.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.349+0000 7f1efb669700 1 -- 172.21.15.3:0/3272932522 wait complete. 2023-11-04T05:29:28.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:29:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 Processor -- start 2023-11-04T05:29:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 -- start start 2023-11-04T05:29:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc40 0x7f636c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f636c0cf930 0x7f636c161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636c06d680 con 0x7f636c0cd7e0 2023-11-04T05:29:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636c0d1510 con 0x7f636c0cf930 2023-11-04T05:29:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.487+0000 7f6371520700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c06d960 con 0x7f636c15dc40 2023-11-04T05:29:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f636c0cd7e0 0x7f636c0cf220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58896/0 (socket says 172.21.15.3:58896) 2023-11-04T05:29:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636a7fc700 1 -- 172.21.15.3:0/1168286762 learned_addr learned my addr 172.21.15.3:0/1168286762 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636affd700 1 --2- 172.21.15.3:0/1168286762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc40 0x7f636c06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f6369ffb700 1 -- 172.21.15.3:0/1168286762 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762379252 0 0) 0x7f636c06d680 con 0x7f636c0cd7e0 2023-11-04T05:29:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636b7fe700 1 --2- 172.21.15.3:0/1168286762 >> v2:172.21.15.188:3300/0 conn(0x7f636c0cf930 0x7f636c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f6369ffb700 1 -- 172.21.15.3:0/1168286762 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63580035b0 con 0x7f636c0cd7e0 2023-11-04T05:29:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f6369ffb700 1 -- 172.21.15.3:0/1168286762 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330799956 0 0) 0x7f63580035b0 con 0x7f636c0cd7e0 2023-11-04T05:29:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636affd700 1 -- 172.21.15.3:0/1168286762 >> v2:172.21.15.188:3300/0 conn(0x7f636c0cf930 msgr2=0x7f636c161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636affd700 1 --2- 172.21.15.3:0/1168286762 >> v2:172.21.15.188:3300/0 conn(0x7f636c0cf930 0x7f636c161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636affd700 1 -- 172.21.15.3:0/1168286762 >> v1:172.21.15.3:6789/0 conn(0x7f636c0cd7e0 legacy=0x7f636c0cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636affd700 1 -- 172.21.15.3:0/1168286762 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c0d1720 con 0x7f636c15dc40 2023-11-04T05:29:28.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.488+0000 7f636affd700 1 --2- 172.21.15.3:0/1168286762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc40 0x7f636c06cad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f635c009840 tx=0x7f635c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547a69f70f9d63a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6369ffb700 1 -- 172.21.15.3:0/1168286762 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354001670 con 0x7f636c15dc40 2023-11-04T05:29:28.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6369ffb700 1 -- 172.21.15.3:0/1168286762 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c004030 con 0x7f636c15dc40 2023-11-04T05:29:28.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6369ffb700 1 -- 172.21.15.3:0/1168286762 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0043c0 con 0x7f636c15dc40 2023-11-04T05:29:28.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 -- 172.21.15.3:0/1168286762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc40 msgr2=0x7f636c06cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 --2- 172.21.15.3:0/1168286762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc40 0x7f636c06cad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f635c009840 tx=0x7f635c009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:28.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 -- 172.21.15.3:0/1168286762 shutdown_connections 2023-11-04T05:29:28.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 --2- 172.21.15.3:0/1168286762 >> v2:172.21.15.188:3300/0 conn(0x7f636c0cf930 0x7f636c161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 --2- 172.21.15.3:0/1168286762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc40 0x7f636c06cad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 -- 172.21.15.3:0/1168286762 >> 172.21.15.3:0/1168286762 conn(0x7f636c06ab20 msgr2=0x7f636c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:28.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 -- 172.21.15.3:0/1168286762 shutdown_connections 2023-11-04T05:29:28.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.489+0000 7f6371520700 1 -- 172.21.15.3:0/1168286762 wait complete. 2023-11-04T05:29:28.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 Processor -- start 2023-11-04T05:29:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 -- start start 2023-11-04T05:29:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 0x7f636c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15dc40 0x7f636c152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0ca440 0x7f636c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c157a20 con 0x7f636c0cf930 2023-11-04T05:29:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c156ea0 con 0x7f636c15dc40 2023-11-04T05:29:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f6371520700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c157180 con 0x7f636c0ca440 2023-11-04T05:29:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.491+0000 7f636affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 0x7f636c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 0x7f636c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53554/0 (socket says 172.21.15.3:53554) 2023-11-04T05:29:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 -- 172.21.15.3:0/37457227 learned_addr learned my addr 172.21.15.3:0/37457227 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636b7fe700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0ca440 0x7f636c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636a7fc700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15dc40 0x7f636c152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 -- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0ca440 msgr2=0x7f636c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0ca440 0x7f636c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 -- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15dc40 msgr2=0x7f636c152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15dc40 0x7f636c152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 -- 172.21.15.3:0/37457227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c0094a0 con 0x7f636c0cf930 2023-11-04T05:29:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f636affd700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 0x7f636c1568c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f635c00a010 tx=0x7f635c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003ae0 con 0x7f636c0cf930 2023-11-04T05:29:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c01f070 con 0x7f636c0cf930 2023-11-04T05:29:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c157460 con 0x7f636c0cf930 2023-11-04T05:29:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003ea0 con 0x7f636c0cf930 2023-11-04T05:29:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.492+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c1530b0 con 0x7f636c0cf930 2023-11-04T05:29:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.493+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c074df0 con 0x7f636c0cf930 2023-11-04T05:29:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.494+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f635c021030 con 0x7f636c0cf930 2023-11-04T05:29:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.494+0000 7f63637fe700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f634c06af00 0x7f634c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.494+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f635c08aaf0 con 0x7f636c0cf930 2023-11-04T05:29:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.494+0000 7f636a7fc700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f634c06af00 0x7f634c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.495+0000 7f636a7fc700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f634c06af00 0x7f634c06d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f636c0b6e10 tx=0x7f6354001230 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.499+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f635c055870 con 0x7f636c0cf930 2023-11-04T05:29:28.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.745+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f636c0c48c0 con 0x7f636c0cf930 2023-11-04T05:29:28.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.746+0000 7f63637fe700 1 -- 172.21.15.3:0/37457227 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f635c058e90 con 0x7f636c0cf930 2023-11-04T05:29:28.748 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:28.748 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:29:28.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f634c06af00 msgr2=0x7f634c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f634c06af00 0x7f634c06d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f636c0b6e10 tx=0x7f6354001230 comp rx=0 tx=0).stop 2023-11-04T05:29:28.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 msgr2=0x7f636c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 0x7f636c1568c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f635c00a010 tx=0x7f635c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 shutdown_connections 2023-11-04T05:29:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f634c06af00 0x7f634c06d3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0ca440 0x7f636c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f636c0cf930 0x7f636c1568c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 --2- 172.21.15.3:0/37457227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15dc40 0x7f636c152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 >> 172.21.15.3:0/37457227 conn(0x7f636c06ab20 msgr2=0x7f636c0cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 shutdown_connections 2023-11-04T05:29:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.748+0000 7f6371520700 1 -- 172.21.15.3:0/37457227 wait complete. 2023-11-04T05:29:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:29:28.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:28.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.886+0000 7f9486a8b700 1 Processor -- start 2023-11-04T05:29:28.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9486a8b700 1 -- start start 2023-11-04T05:29:28.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9486a8b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 0x7f94800728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9486a8b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9480072df0 con 0x7f948006ab20 2023-11-04T05:29:28.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9486a8b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94800cda00 con 0x7f948006af00 2023-11-04T05:29:28.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9486a8b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94800cf050 con 0x7f94800cc7a0 2023-11-04T05:29:28.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9484827700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f948006ab20 0x7f94800cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58910/0 (socket says 172.21.15.3:58910) 2023-11-04T05:29:28.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.887+0000 7f9484827700 1 -- 172.21.15.3:0/1335601047 learned_addr learned my addr 172.21.15.3:0/1335601047 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:28.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947ffff700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 0x7f94800728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924525763 0 0) 0x7f9480072df0 con 0x7f948006ab20 2023-11-04T05:29:28.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94700035b0 con 0x7f948006ab20 2023-11-04T05:29:28.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1301881047 0 0) 0x7f94700035b0 con 0x7f948006ab20 2023-11-04T05:29:28.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 msgr2=0x7f94800728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:28.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 0x7f94800728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 >> v1:172.21.15.188:6789/0 conn(0x7f948006af00 legacy=0x7f94800d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800cdce0 con 0x7f948006ab20 2023-11-04T05:29:28.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f9474001670 con 0x7f948006ab20 2023-11-04T05:29:28.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 >> v1:172.21.15.3:6789/0 conn(0x7f948006ab20 legacy=0x7f94800cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f9470003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94700043e0 0x7f94700048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 0x7f9470005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94800cf050 con 0x7f9470003a30 2023-11-04T05:29:28.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947000ba10 con 0x7f94700043e0 2023-11-04T05:29:28.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947000bbd0 con 0x7f94800cc7a0 2023-11-04T05:29:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f9484827700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f9470003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f947ffff700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94700043e0 0x7f94700048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.888+0000 7f9485028700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 0x7f9470005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9484827700 1 -- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 msgr2=0x7f9470005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9484827700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800cc7a0 0x7f9470005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9484827700 1 -- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94700043e0 msgr2=0x7f94700048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9484827700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94700043e0 0x7f94700048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9484827700 1 -- 172.21.15.3:0/1335601047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700035b0 con 0x7f9470003a30 2023-11-04T05:29:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9484827700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f9470003e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f947400c240 tx=0x7f9470005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1dc879c835e4fa server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947400cd80 con 0x7f9470003a30 2023-11-04T05:29:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9474005960 con 0x7f9470003a30 2023-11-04T05:29:28.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f947f7fe700 1 -- 172.21.15.3:0/1335601047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474005d10 con 0x7f9470003a30 2023-11-04T05:29:28.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9486a8b700 1 -- 172.21.15.3:0/1335601047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 msgr2=0x7f9470003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9486a8b700 1 --2- 172.21.15.3:0/1335601047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f9470003e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f947400c240 tx=0x7f9470005880 comp rx=0 tx=0).stop 2023-11-04T05:29:28.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9485028700 1 -- 172.21.15.3:0/1335601047 reap_dead start 2023-11-04T05:29:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9486a8b700 1 -- 172.21.15.3:0/1335601047 shutdown_connections 2023-11-04T05:29:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9486a8b700 1 -- 172.21.15.3:0/1335601047 >> 172.21.15.3:0/1335601047 conn(0x7f948006df90 msgr2=0x7f948006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9486a8b700 1 -- 172.21.15.3:0/1335601047 shutdown_connections 2023-11-04T05:29:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.889+0000 7f9486a8b700 1 -- 172.21.15.3:0/1335601047 wait complete. 2023-11-04T05:29:28.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.891+0000 7f9486a8b700 1 Processor -- start 2023-11-04T05:29:28.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.891+0000 7f9486a8b700 1 -- start start 2023-11-04T05:29:28.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9486a8b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f94801e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9486a8b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94801573a0 0x7f94800c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9486a8b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800c4fc0 0x7f94800c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9486a8b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9480154fc0 con 0x7f9470003a30 2023-11-04T05:29:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9486a8b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94801e9270 con 0x7f94800c4fc0 2023-11-04T05:29:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9486a8b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94801e9550 con 0x7f94801573a0 2023-11-04T05:29:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9484827700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f94801e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9484827700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f94801e65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53574/0 (socket says 172.21.15.3:53574) 2023-11-04T05:29:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.892+0000 7f9484827700 1 -- 172.21.15.3:0/3526191388 learned_addr learned my addr 172.21.15.3:0/3526191388 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f947ffff700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94801573a0 0x7f94800c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9485028700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800c4fc0 0x7f94800c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9484827700 1 -- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94801573a0 msgr2=0x7f94800c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9484827700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94801573a0 0x7f94800c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9484827700 1 -- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800c4fc0 msgr2=0x7f94800c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9484827700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800c4fc0 0x7f94800c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9484827700 1 -- 172.21.15.3:0/3526191388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94801e9830 con 0x7f9470003a30 2023-11-04T05:29:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9485028700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800c4fc0 0x7f94800c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9484827700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f94801e65b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9474005d80 tx=0x7f9474001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474004c80 con 0x7f9470003a30 2023-11-04T05:29:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9474004de0 con 0x7f9470003a30 2023-11-04T05:29:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9480155210 con 0x7f9470003a30 2023-11-04T05:29:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474007310 con 0x7f9470003a30 2023-11-04T05:29:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.893+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9480157820 con 0x7f9470003a30 2023-11-04T05:29:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.894+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9480155580 con 0x7f9470003a30 2023-11-04T05:29:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.894+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9474019070 con 0x7f9470003a30 2023-11-04T05:29:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.895+0000 7f947d7fa700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f946406af00 0x7f946406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.895+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f947408a890 con 0x7f9470003a30 2023-11-04T05:29:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.895+0000 7f947ffff700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f946406af00 0x7f946406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.896+0000 7f947ffff700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f946406af00 0x7f946406d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f946c009aa0 tx=0x7f946c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:28.900+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f9474055640 con 0x7f9470003a30 2023-11-04T05:29:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.104+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9480157a10 con 0x7f9470003a30 2023-11-04T05:29:29.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.105+0000 7f947d7fa700 1 -- 172.21.15.3:0/3526191388 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7f9474058c60 con 0x7f9470003a30 2023-11-04T05:29:29.107 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:29.107 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f946406af00 msgr2=0x7f946406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f946406af00 0x7f946406d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f946c009aa0 tx=0x7f946c008040 comp rx=0 tx=0).stop 2023-11-04T05:29:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 msgr2=0x7f94801e65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f94801e65b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9474005d80 tx=0x7f9474001980 comp rx=0 tx=0).stop 2023-11-04T05:29:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 shutdown_connections 2023-11-04T05:29:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f946406af00 0x7f946406d3c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800c4fc0 0x7f94800c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9470003a30 0x7f94801e65b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 --2- 172.21.15.3:0/3526191388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94801573a0 0x7f94800c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 >> 172.21.15.3:0/3526191388 conn(0x7f948006df90 msgr2=0x7f9480160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:29.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 shutdown_connections 2023-11-04T05:29:29.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.107+0000 7f9486a8b700 1 -- 172.21.15.3:0/3526191388 wait complete. 2023-11-04T05:29:29.121 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-04T05:29:29.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:29:29.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.246+0000 7f353d9a3700 1 Processor -- start 2023-11-04T05:29:29.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.246+0000 7f353d9a3700 1 -- start start 2023-11-04T05:29:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f353d9a3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35380d2690 0x7f353806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f353d9a3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35380d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f353d9a3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35381616a0 con 0x7f35381612c0 2023-11-04T05:29:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f353d9a3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35380b2bb0 con 0x7f35380d2690 2023-11-04T05:29:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f353d9a3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35380d42c0 con 0x7f35380b2e00 2023-11-04T05:29:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f3536ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f35381612c0 0x7f35380d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58930/0 (socket says 172.21.15.3:58930) 2023-11-04T05:29:29.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f3536ffd700 1 -- 172.21.15.3:0/3297315539 learned_addr learned my addr 172.21.15.3:0/3297315539 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f35367fc700 1 --2- 172.21.15.3:0/3297315539 >> v2:172.21.15.188:3300/0 conn(0x7f35380d2690 0x7f353806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f35377fe700 1 --2- 172.21.15.3:0/3297315539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35380d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f3535ffb700 1 -- 172.21.15.3:0/3297315539 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462714271 0 0) 0x7f35381616a0 con 0x7f35381612c0 2023-11-04T05:29:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.247+0000 7f3535ffb700 1 -- 172.21.15.3:0/3297315539 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35200035b0 con 0x7f35381612c0 2023-11-04T05:29:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f35377fe700 1 -- 172.21.15.3:0/3297315539 >> v2:172.21.15.188:3300/0 conn(0x7f35380d2690 msgr2=0x7f353806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:29.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f35377fe700 1 --2- 172.21.15.3:0/3297315539 >> v2:172.21.15.188:3300/0 conn(0x7f35380d2690 0x7f353806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f35377fe700 1 -- 172.21.15.3:0/3297315539 >> v1:172.21.15.3:6789/0 conn(0x7f35381612c0 legacy=0x7f35380d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f35377fe700 1 -- 172.21.15.3:0/3297315539 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35380d45a0 con 0x7f35380b2e00 2023-11-04T05:29:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f35367fc700 1 --2- 172.21.15.3:0/3297315539 >> v2:172.21.15.188:3300/0 conn(0x7f35380d2690 0x7f353806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T05:29:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f35377fe700 1 --2- 172.21.15.3:0/3297315539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35380d1780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3530009d90 tx=0x7f353000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61673a087a8acc1d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f3535ffb700 1 -- 172.21.15.3:0/3297315539 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353000e070 con 0x7f35380b2e00 2023-11-04T05:29:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f3535ffb700 1 -- 172.21.15.3:0/3297315539 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353000bb20 con 0x7f35380b2e00 2023-11-04T05:29:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.248+0000 7f3535ffb700 1 -- 172.21.15.3:0/3297315539 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530012430 con 0x7f35380b2e00 2023-11-04T05:29:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 -- 172.21.15.3:0/3297315539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 msgr2=0x7f35380d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3297315539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35380d1780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3530009d90 tx=0x7f353000b330 comp rx=0 tx=0).stop 2023-11-04T05:29:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 -- 172.21.15.3:0/3297315539 shutdown_connections 2023-11-04T05:29:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3297315539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35380d1780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3297315539 >> v2:172.21.15.188:3300/0 conn(0x7f35380d2690 0x7f353806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 -- 172.21.15.3:0/3297315539 >> 172.21.15.3:0/3297315539 conn(0x7f353808aff0 msgr2=0x7f35380d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 -- 172.21.15.3:0/3297315539 shutdown_connections 2023-11-04T05:29:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.249+0000 7f353d9a3700 1 -- 172.21.15.3:0/3297315539 wait complete. 2023-11-04T05:29:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.250+0000 7f353d9a3700 1 Processor -- start 2023-11-04T05:29:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.250+0000 7f353d9a3700 1 -- start start 2023-11-04T05:29:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f353d9a3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35381f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f353d9a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 0x7f353815fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f353d9a3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35381612c0 0x7f35380b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f353d9a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35380b8ad0 con 0x7f35380d2690 2023-11-04T05:29:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f353d9a3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35380b7e50 con 0x7f35381612c0 2023-11-04T05:29:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f353d9a3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35380b8130 con 0x7f35380b2e00 2023-11-04T05:29:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 0x7f353815fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 0x7f353815fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53590/0 (socket says 172.21.15.3:53590) 2023-11-04T05:29:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 -- 172.21.15.3:0/3431118728 learned_addr learned my addr 172.21.15.3:0/3431118728 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f3536ffd700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35381f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35377fe700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35381612c0 0x7f35380b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 -- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 msgr2=0x7f35381f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35381f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 -- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35381612c0 msgr2=0x7f35380b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35381612c0 0x7f35380b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.251+0000 7f35367fc700 1 -- 172.21.15.3:0/3431118728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280094a0 con 0x7f35380d2690 2023-11-04T05:29:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.252+0000 7f35367fc700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 0x7f353815fb40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f352800ebe0 tx=0x7f352800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.252+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528009760 con 0x7f35380d2690 2023-11-04T05:29:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.252+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352800c850 con 0x7f35380d2690 2023-11-04T05:29:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.252+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3530009a70 con 0x7f35380d2690 2023-11-04T05:29:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.252+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800cbe0 con 0x7f35380d2690 2023-11-04T05:29:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.252+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35380b86f0 con 0x7f35380d2690 2023-11-04T05:29:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.253+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3538074df0 con 0x7f35380d2690 2023-11-04T05:29:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.253+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f352800cd40 con 0x7f35380d2690 2023-11-04T05:29:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.254+0000 7f351ffff700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f352406af00 0x7f352406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.254+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f35280898c0 con 0x7f35380d2690 2023-11-04T05:29:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.254+0000 7f3536ffd700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f352406af00 0x7f352406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.254+0000 7f3536ffd700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f352406af00 0x7f352406d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f352c0010e0 tx=0x7f352c007c70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.258+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f3528054640 con 0x7f35380d2690 2023-11-04T05:29:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.504+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35380d1100 con 0x7f35380d2690 2023-11-04T05:29:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.504+0000 7f351ffff700 1 -- 172.21.15.3:0/3431118728 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f3528057c60 con 0x7f35380d2690 2023-11-04T05:29:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:29.506 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:29:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.506+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f352406af00 msgr2=0x7f352406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.506+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f352406af00 0x7f352406d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f352c0010e0 tx=0x7f352c007c70 comp rx=0 tx=0).stop 2023-11-04T05:29:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.506+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 msgr2=0x7f353815fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.506+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 0x7f353815fb40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f352800ebe0 tx=0x7f352800eef0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 shutdown_connections 2023-11-04T05:29:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f352406af00 0x7f352406d3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35381612c0 0x7f35380b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35380b2e00 0x7f35381f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 --2- 172.21.15.3:0/3431118728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35380d2690 0x7f353815fb40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 >> 172.21.15.3:0/3431118728 conn(0x7f353808aff0 msgr2=0x7f35380d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 shutdown_connections 2023-11-04T05:29:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.507+0000 7f353d9a3700 1 -- 172.21.15.3:0/3431118728 wait complete. 2023-11-04T05:29:29.510 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:29:29.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:29:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.644+0000 7fe6c1f7d700 1 Processor -- start 2023-11-04T05:29:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.644+0000 7fe6c1f7d700 1 -- start start 2023-11-04T05:29:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6c1f7d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 0x7fe6bc0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6c1f7d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0d0570 con 0x7fe6bc0b5f30 2023-11-04T05:29:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6c1f7d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0cab90 con 0x7fe6bc06ab20 2023-11-04T05:29:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6c1f7d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0d13f0 con 0x7fe6bc0d0a20 2023-11-04T05:29:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6baffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe6bc0b5f30 0x7fe6bc0b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58946/0 (socket says 172.21.15.3:58946) 2023-11-04T05:29:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6baffd700 1 -- 172.21.15.3:0/1890803927 learned_addr learned my addr 172.21.15.3:0/1890803927 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 0x7fe6bc0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253298709 0 0) 0x7fe6bc0d0570 con 0x7fe6bc0b5f30 2023-11-04T05:29:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a80035b0 con 0x7fe6bc0b5f30 2023-11-04T05:29:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.645+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186544882 0 0) 0x7fe6a80035b0 con 0x7fe6bc0b5f30 2023-11-04T05:29:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 msgr2=0x7fe6bc0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 0x7fe6bc0d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 >> v1:172.21.15.188:6789/0 conn(0x7fe6bc06ab20 legacy=0x7fe6bc06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0cae70 con 0x7fe6bc0b5f30 2023-11-04T05:29:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fe6a40017f0 con 0x7fe6bc0b5f30 2023-11-04T05:29:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 >> v1:172.21.15.3:6789/0 conn(0x7fe6bc0b5f30 legacy=0x7fe6bc0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 0x7fe6a8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a8003f70 0x7fe6a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6a8004d90 0x7fe6a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0d13f0 con 0x7fe6a8004d90 2023-11-04T05:29:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800ba70 con 0x7fe6a8003f70 2023-11-04T05:29:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800bc30 con 0x7fe6bc0d0a20 2023-11-04T05:29:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6a8004d90 0x7fe6a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bb7fe700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 0x7fe6a8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6baffd700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a8003f70 0x7fe6a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 -- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 msgr2=0x7fe6a8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0d0a20 0x7fe6a8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 -- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a8003f70 msgr2=0x7fe6a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a8003f70 0x7fe6a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 -- 172.21.15.3:0/1890803927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80035b0 con 0x7fe6a8004d90 2023-11-04T05:29:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6baffd700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6a8003f70 0x7fe6a8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6a8004d90 0x7fe6a8005210 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009fc0 tx=0x7fe6a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9824078e6a4a824b server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400ec30 con 0x7fe6a8004d90 2023-11-04T05:29:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b400f040 con 0x7fe6a8004d90 2023-11-04T05:29:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.646+0000 7fe6ba7fc700 1 -- 172.21.15.3:0/1890803927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4014480 con 0x7fe6a8004d90 2023-11-04T05:29:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/1890803927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6a8004d90 msgr2=0x7fe6a8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/1890803927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6a8004d90 0x7fe6a8005210 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009fc0 tx=0x7fe6a8005880 comp rx=0 tx=0).stop 2023-11-04T05:29:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6bbfff700 1 -- 172.21.15.3:0/1890803927 reap_dead start 2023-11-04T05:29:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/1890803927 shutdown_connections 2023-11-04T05:29:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/1890803927 >> 172.21.15.3:0/1890803927 conn(0x7fe6bc06df90 msgr2=0x7fe6bc06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/1890803927 shutdown_connections 2023-11-04T05:29:29.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.647+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/1890803927 wait complete. 2023-11-04T05:29:29.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.648+0000 7fe6c1f7d700 1 Processor -- start 2023-11-04T05:29:29.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.648+0000 7fe6c1f7d700 1 -- start start 2023-11-04T05:29:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6c1f7d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6bc1599e0 0x7fe6bc0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6c1f7d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0c67d0 0x7fe6bc0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6c1f7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 0x7fe6bc1f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6c1f7d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0c4de0 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6c1f7d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc15cfa0 con 0x7fe6bc1599e0 2023-11-04T05:29:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6c1f7d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc15d280 con 0x7fe6bc0c67d0 2023-11-04T05:29:29.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 0x7fe6bc1f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 0x7fe6bc1f6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53610/0 (socket says 172.21.15.3:53610) 2023-11-04T05:29:29.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 -- 172.21.15.3:0/747750606 learned_addr learned my addr 172.21.15.3:0/747750606 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:29.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6baffd700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0c67d0 0x7fe6bc0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bb7fe700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6bc1599e0 0x7fe6bc0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 -- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0c67d0 msgr2=0x7fe6bc0c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0c67d0 0x7fe6bc0c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 -- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6bc1599e0 msgr2=0x7fe6bc0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.649+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6bc1599e0 0x7fe6bc0c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6bbfff700 1 -- 172.21.15.3:0/747750606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc15d560 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6bbfff700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 0x7fe6bc1f6370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4016000 tx=0x7fe6b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:29.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4007000 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4022070 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc15d840 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4029480 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.650+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc0c4a60 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.651+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc15da30 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.651+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6b401e430 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.652+0000 7fe6b3fff700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe69c06af00 0x7fe69c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:29.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.652+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe6b4024070 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.652+0000 7fe6bb7fe700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe69c06af00 0x7fe69c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:29.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.653+0000 7fe6bb7fe700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe69c06af00 0x7fe69c06d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0011a0 tx=0x7fe6ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:29.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.657+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fe6b4056840 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.906+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6bc074df0 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.906+0000 7fe6b3fff700 1 -- 172.21.15.3:0/747750606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7fe6b4059e60 con 0x7fe6bc0c45b0 2023-11-04T05:29:29.908 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:29.908 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2510096827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2510096827},{"type":"v1","addr":"172.21.15.3:6833","nonce":2510096827}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2575363405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2575363405},{"type":"v1","addr":"172.21.15.188:6836","nonce":2575363405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/1061547584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1061547584},{"type":"v1","addr":"172.21.15.3:6835","nonce":1061547584}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:28:20.853748+0000","modified":"2023-11-04T05:28:44.290398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3609354532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3609354532},{"type":"v1","addr":"172.21.15.188:6837","nonce":3609354532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-04T05:29:29.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.908+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe69c06af00 msgr2=0x7fe69c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.908+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe69c06af00 0x7fe69c06d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0011a0 tx=0x7fe6ac009040 comp rx=0 tx=0).stop 2023-11-04T05:29:29.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.908+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 msgr2=0x7fe6bc1f6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.908+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 0x7fe6bc1f6370 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4016000 tx=0x7fe6b4009750 comp rx=0 tx=0).stop 2023-11-04T05:29:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 shutdown_connections 2023-11-04T05:29:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe69c06af00 0x7fe69c06d3c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6bc1599e0 0x7fe6bc0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6bc0c45b0 0x7fe6bc1f6370 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 --2- 172.21.15.3:0/747750606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6bc0c67d0 0x7fe6bc0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 >> 172.21.15.3:0/747750606 conn(0x7fe6bc06df90 msgr2=0x7fe6bc0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 shutdown_connections 2023-11-04T05:29:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:29.909+0000 7fe6c1f7d700 1 -- 172.21.15.3:0/747750606 wait complete. 2023-11-04T05:29:29.913 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-11-04T05:29:29.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.047+0000 7fcaca8cb700 1 Processor -- start 2023-11-04T05:29:30.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.047+0000 7fcaca8cb700 1 -- start start 2023-11-04T05:29:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcaca8cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcac40d2690 0x7fcac406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcaca8cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac40d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcaca8cb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac41616a0 con 0x7fcac41612c0 2023-11-04T05:29:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcaca8cb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac40b2bb0 con 0x7fcac40d2690 2023-11-04T05:29:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcaca8cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40d42c0 con 0x7fcac40b2e00 2023-11-04T05:29:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcac40d2690 0x7fcac406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcac40d2690 0x7fcac406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53618/0 (socket says 172.21.15.3:53618) 2023-11-04T05:29:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 -- 172.21.15.3:0/647965536 learned_addr learned my addr 172.21.15.3:0/647965536 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac8e68700 1 --2- 172.21.15.3:0/647965536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac40d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 -- 172.21.15.3:0/647965536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 msgr2=0x7fcac40d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 --2- 172.21.15.3:0/647965536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac40d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 -- 172.21.15.3:0/647965536 >> v1:172.21.15.188:6789/0 conn(0x7fcac41612c0 legacy=0x7fcac40d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.048+0000 7fcac37fe700 1 -- 172.21.15.3:0/647965536 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac40d45a0 con 0x7fcac40d2690 2023-11-04T05:29:30.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcac37fe700 1 --2- 172.21.15.3:0/647965536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac406bd30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcab80098f0 tx=0x7fcab8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d397331508dbec99 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcac2ffd700 1 -- 172.21.15.3:0/647965536 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8011830 con 0x7fcac40d2690 2023-11-04T05:29:30.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcac2ffd700 1 -- 172.21.15.3:0/647965536 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8011990 con 0x7fcac40d2690 2023-11-04T05:29:30.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcac2ffd700 1 -- 172.21.15.3:0/647965536 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8011d20 con 0x7fcac40d2690 2023-11-04T05:29:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 -- 172.21.15.3:0/647965536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 msgr2=0x7fcac406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/647965536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac406bd30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcab80098f0 tx=0x7fcab8009c00 comp rx=0 tx=0).stop 2023-11-04T05:29:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 -- 172.21.15.3:0/647965536 shutdown_connections 2023-11-04T05:29:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/647965536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac40d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/647965536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac406bd30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 -- 172.21.15.3:0/647965536 >> 172.21.15.3:0/647965536 conn(0x7fcac408aff0 msgr2=0x7fcac40d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 -- 172.21.15.3:0/647965536 shutdown_connections 2023-11-04T05:29:30.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.049+0000 7fcaca8cb700 1 -- 172.21.15.3:0/647965536 wait complete. 2023-11-04T05:29:30.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.051+0000 7fcaca8cb700 1 Processor -- start 2023-11-04T05:29:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.051+0000 7fcaca8cb700 1 -- start start 2023-11-04T05:29:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.051+0000 7fcaca8cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac4158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.051+0000 7fcaca8cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac4152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.051+0000 7fcaca8cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac41612c0 0x7fcac4153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcaca8cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4159be0 con 0x7fcac40d2690 2023-11-04T05:29:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcaca8cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4158b40 con 0x7fcac41612c0 2023-11-04T05:29:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcaca8cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4158e20 con 0x7fcac40b2e00 2023-11-04T05:29:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac4152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac4152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53622/0 (socket says 172.21.15.3:53622) 2023-11-04T05:29:30.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 -- 172.21.15.3:0/1718222078 learned_addr learned my addr 172.21.15.3:0/1718222078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac3fff700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac4158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 -- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 msgr2=0x7fcac4158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac8e68700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac41612c0 0x7fcac4153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac4158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 -- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac41612c0 msgr2=0x7fcac4153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac41612c0 0x7fcac4153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 -- 172.21.15.3:0/1718222078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab8009550 con 0x7fcac40d2690 2023-11-04T05:29:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac37fe700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac4152b00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009c40 tx=0x7fcab8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80036a0 con 0x7fcac40d2690 2023-11-04T05:29:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab801e070 con 0x7fcac40d2690 2023-11-04T05:29:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8003a60 con 0x7fcac40d2690 2023-11-04T05:29:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac4159100 con 0x7fcac40d2690 2023-11-04T05:29:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.052+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac41596c0 con 0x7fcac40d2690 2023-11-04T05:29:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.053+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac40b55d0 con 0x7fcac40d2690 2023-11-04T05:29:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.054+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcab8003bc0 con 0x7fcac40d2690 2023-11-04T05:29:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.054+0000 7fcac0ff9700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcab006af00 0x7fcab006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.054+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcab808a890 con 0x7fcac40d2690 2023-11-04T05:29:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.054+0000 7fcac3fff700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcab006af00 0x7fcab006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.055+0000 7fcac3fff700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcab006af00 0x7fcab006d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000fb0 tx=0x7fcab4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.059+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fcab8055610 con 0x7fcac40d2690 2023-11-04T05:29:30.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.258+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcac41598b0 con 0x7fcac40d2690 2023-11-04T05:29:30.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.259+0000 7fcac0ff9700 1 -- 172.21.15.3:0/1718222078 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18674 (secure 0 0 0) 0x7fcab8020090 con 0x7fcac40d2690 2023-11-04T05:29:30.260 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:30.261 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:04.969904+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:28:18.812474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-04T05:28:19.827692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-04T05:28:23.853174+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:30.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcab006af00 msgr2=0x7fcab006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcab006af00 0x7fcab006d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000fb0 tx=0x7fcab4009040 comp rx=0 tx=0).stop 2023-11-04T05:29:30.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 msgr2=0x7fcac4152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac4152b00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009c40 tx=0x7fcab8004750 comp rx=0 tx=0).stop 2023-11-04T05:29:30.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 shutdown_connections 2023-11-04T05:29:30.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcab006af00 0x7fcab006d3c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac41612c0 0x7fcac4153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40b2e00 0x7fcac4158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 --2- 172.21.15.3:0/1718222078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac40d2690 0x7fcac4152b00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 >> 172.21.15.3:0/1718222078 conn(0x7fcac408aff0 msgr2=0x7fcac40b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:30.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 shutdown_connections 2023-11-04T05:29:30.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.261+0000 7fcaca8cb700 1 -- 172.21.15.3:0/1718222078 wait complete. 2023-11-04T05:29:30.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-04T05:29:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.400+0000 7f5c2ed3c700 1 Processor -- start 2023-11-04T05:29:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.400+0000 7f5c2ed3c700 1 -- start start 2023-11-04T05:29:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2ed3c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c280ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2ed3c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c280cc7a0 0x7f5c280ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2ed3c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280cd0a0 con 0x7f5c280b4a40 2023-11-04T05:29:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2ed3c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280d0240 con 0x7f5c280cc7a0 2023-11-04T05:29:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2ed3c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280ceac0 con 0x7f5c2815be70 2023-11-04T05:29:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c280cc7a0 0x7f5c280ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c280cc7a0 0x7f5c280ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53634/0 (socket says 172.21.15.3:53634) 2023-11-04T05:29:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 -- 172.21.15.3:0/2824243507 learned_addr learned my addr 172.21.15.3:0/2824243507 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 -- 172.21.15.3:0/2824243507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 msgr2=0x7f5c280ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2cad8700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c280ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c280ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 -- 172.21.15.3:0/2824243507 >> v1:172.21.15.188:6789/0 conn(0x7f5c280b4a40 legacy=0x7f5c280cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 -- 172.21.15.3:0/2824243507 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280ccdc0 con 0x7f5c280cc7a0 2023-11-04T05:29:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2cad8700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c280ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c2d2d9700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280cc7a0 0x7f5c280ce580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009e50 tx=0x7f5c2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b188d9ce5906647d server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.401+0000 7f5c277fe700 1 -- 172.21.15.3:0/2824243507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000e070 con 0x7f5c280cc7a0 2023-11-04T05:29:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c277fe700 1 -- 172.21.15.3:0/2824243507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20003a20 con 0x7f5c280cc7a0 2023-11-04T05:29:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c277fe700 1 -- 172.21.15.3:0/2824243507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20003dd0 con 0x7f5c280cc7a0 2023-11-04T05:29:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/2824243507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280cc7a0 msgr2=0x7f5c280ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280cc7a0 0x7f5c280ce580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009e50 tx=0x7f5c2000b330 comp rx=0 tx=0).stop 2023-11-04T05:29:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/2824243507 shutdown_connections 2023-11-04T05:29:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280cc7a0 0x7f5c280ce580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/2824243507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c280ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/2824243507 >> 172.21.15.3:0/2824243507 conn(0x7f5c2806f0a0 msgr2=0x7f5c2806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/2824243507 shutdown_connections 2023-11-04T05:29:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.402+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/2824243507 wait complete. 2023-11-04T05:29:30.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.403+0000 7f5c2ed3c700 1 Processor -- start 2023-11-04T05:29:30.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.403+0000 7f5c2ed3c700 1 -- start start 2023-11-04T05:29:30.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2ed3c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 0x7f5c2806bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2ed3c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280cc7a0 0x7f5c280c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2ed3c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c28153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2ed3c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280d1b50 con 0x7f5c280b4a40 2023-11-04T05:29:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2ed3c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2806dc40 con 0x7f5c280cc7a0 2023-11-04T05:29:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2ed3c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280c70c0 con 0x7f5c2815be70 2023-11-04T05:29:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2cad8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 0x7f5c2806bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2cad8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 0x7f5c2806bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53636/0 (socket says 172.21.15.3:53636) 2023-11-04T05:29:30.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.404+0000 7f5c2cad8700 1 -- 172.21.15.3:0/1663716421 learned_addr learned my addr 172.21.15.3:0/1663716421 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c27fff700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280cc7a0 0x7f5c280c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2d2d9700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c28153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2cad8700 1 -- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 msgr2=0x7f5c28153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2cad8700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c28153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2cad8700 1 -- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280cc7a0 msgr2=0x7f5c280c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2cad8700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280cc7a0 0x7f5c280c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2cad8700 1 -- 172.21.15.3:0/1663716421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20009b30 con 0x7f5c280b4a40 2023-11-04T05:29:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c27fff700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280cc7a0 0x7f5c280c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2d2d9700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c28153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2cad8700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 0x7f5c2806bc30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c7f0 tx=0x7f5c1c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c011800 con 0x7f5c280b4a40 2023-11-04T05:29:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c011960 con 0x7f5c280b4a40 2023-11-04T05:29:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c280c73a0 con 0x7f5c280b4a40 2023-11-04T05:29:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c011d10 con 0x7f5c280b4a40 2023-11-04T05:29:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.405+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280d2080 con 0x7f5c280b4a40 2023-11-04T05:29:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.406+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c280c75e0 con 0x7f5c280b4a40 2023-11-04T05:29:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.407+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c1c018360 con 0x7f5c280b4a40 2023-11-04T05:29:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.407+0000 7f5c257fa700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c0c06af00 0x7f5c0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.407+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5c1c0899e0 con 0x7f5c280b4a40 2023-11-04T05:29:30.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.407+0000 7f5c27fff700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c0c06af00 0x7f5c0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.408+0000 7f5c27fff700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c0c06af00 0x7f5c0c06d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5c28156ef0 tx=0x7f5c14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.411+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f5c1c054650 con 0x7f5c280b4a40 2023-11-04T05:29:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.658+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5c2815cf50 con 0x7f5c280b4a40 2023-11-04T05:29:30.660 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-11-04T05:29:30.658+0000 7f99ea120700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-04T05:29:30.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.660+0000 7f5c257fa700 1 -- 172.21.15.3:0/1663716421 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5c1c057c70 con 0x7f5c280b4a40 2023-11-04T05:29:30.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c0c06af00 msgr2=0x7f5c0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c0c06af00 0x7f5c0c06d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5c28156ef0 tx=0x7f5c14009040 comp rx=0 tx=0).stop 2023-11-04T05:29:30.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 msgr2=0x7f5c2806bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 0x7f5c2806bc30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c7f0 tx=0x7f5c1c00cbb0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 shutdown_connections 2023-11-04T05:29:30.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f5c0c06af00 0x7f5c0c06d3c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c2815be70 0x7f5c28153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c280b4a40 0x7f5c2806bc30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 --2- 172.21.15.3:0/1663716421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280cc7a0 0x7f5c280c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 >> 172.21.15.3:0/1663716421 conn(0x7f5c2806f0a0 msgr2=0x7f5c281595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:30.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 shutdown_connections 2023-11-04T05:29:30.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.663+0000 7f5c2ed3c700 1 -- 172.21.15.3:0/1663716421 wait complete. 2023-11-04T05:29:30.667 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-04T05:29:30.669 INFO:tasks.ceph.mds.d.smithi188.stderr: -5847> 2023-11-04T05:28:27.349+0000 7fcaeebe5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-04T05:29:30.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-04T05:29:30.701 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-04T05:29:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.802+0000 7fde176e3700 1 Processor -- start 2023-11-04T05:29:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.802+0000 7fde176e3700 1 -- start start 2023-11-04T05:29:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde176e3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde1006afa0 0x7fde100d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde176e3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100caca0 con 0x7fde1006ab20 2023-11-04T05:29:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde176e3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100cfe40 con 0x7fde101611d0 2023-11-04T05:29:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde176e3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde100ce0a0 con 0x7fde1006afa0 2023-11-04T05:29:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde15c80700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde1006afa0 0x7fde100d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde14c7e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fde1006ab20 0x7fde100d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58994/0 (socket says 172.21.15.3:58994) 2023-11-04T05:29:30.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde15c80700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde1006afa0 0x7fde100d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53648/0 (socket says 172.21.15.3:53648) 2023-11-04T05:29:30.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde14c7e700 1 -- 172.21.15.3:0/1694351448 learned_addr learned my addr 172.21.15.3:0/1694351448 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:30.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde15c80700 1 -- 172.21.15.3:0/1694351448 >> v1:172.21.15.188:6789/0 conn(0x7fde101611d0 legacy=0x7fde100ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:30.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde15c80700 1 -- 172.21.15.3:0/1694351448 >> v1:172.21.15.3:6789/0 conn(0x7fde1006ab20 legacy=0x7fde100d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde15c80700 1 -- 172.21.15.3:0/1694351448 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100cf850 con 0x7fde1006afa0 2023-11-04T05:29:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.803+0000 7fde07fff700 1 -- 172.21.15.3:0/1694351448 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903188152 0 0) 0x7fde100caca0 con 0x7fde1006ab20 2023-11-04T05:29:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde15c80700 1 --2- 172.21.15.3:0/1694351448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde1006afa0 0x7fde100d09e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fde08009d90 tx=0x7fde0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b933bc470268ae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde07fff700 1 -- 172.21.15.3:0/1694351448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0800e070 con 0x7fde1006afa0 2023-11-04T05:29:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde07fff700 1 -- 172.21.15.3:0/1694351448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800bb90 con 0x7fde1006afa0 2023-11-04T05:29:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde07fff700 1 -- 172.21.15.3:0/1694351448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08003a20 con 0x7fde1006afa0 2023-11-04T05:29:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 -- 172.21.15.3:0/1694351448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde1006afa0 msgr2=0x7fde100d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 --2- 172.21.15.3:0/1694351448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde1006afa0 0x7fde100d09e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fde08009d90 tx=0x7fde0800b330 comp rx=0 tx=0).stop 2023-11-04T05:29:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 -- 172.21.15.3:0/1694351448 shutdown_connections 2023-11-04T05:29:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 --2- 172.21.15.3:0/1694351448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde1006afa0 0x7fde100d09e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 -- 172.21.15.3:0/1694351448 >> 172.21.15.3:0/1694351448 conn(0x7fde10072560 msgr2=0x7fde10099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 -- 172.21.15.3:0/1694351448 shutdown_connections 2023-11-04T05:29:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.804+0000 7fde176e3700 1 -- 172.21.15.3:0/1694351448 wait complete. 2023-11-04T05:29:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 Processor -- start 2023-11-04T05:29:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 -- start start 2023-11-04T05:29:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1006ab20 0x7fde100c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1006afa0 0x7fde100c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 0x7fde101f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde101f6ea0 con 0x7fde101611d0 2023-11-04T05:29:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100c4760 con 0x7fde1006afa0 2023-11-04T05:29:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.806+0000 7fde176e3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100c4a40 con 0x7fde1006ab20 2023-11-04T05:29:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 0x7fde101f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 0x7fde101f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53654/0 (socket says 172.21.15.3:53654) 2023-11-04T05:29:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 -- 172.21.15.3:0/2719681621 learned_addr learned my addr 172.21.15.3:0/2719681621 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde1547f700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1006ab20 0x7fde100c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde14c7e700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1006afa0 0x7fde100c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 -- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1006ab20 msgr2=0x7fde100c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1006ab20 0x7fde100c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 -- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1006afa0 msgr2=0x7fde100c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1006afa0 0x7fde100c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:30.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 -- 172.21.15.3:0/2719681621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08009a70 con 0x7fde101611d0 2023-11-04T05:29:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde14c7e700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1006afa0 0x7fde100c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde15c80700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 0x7fde101f72c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fde100d1750 tx=0x7fde08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08012630 con 0x7fde101611d0 2023-11-04T05:29:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08012790 con 0x7fde101611d0 2023-11-04T05:29:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde100c4d20 con 0x7fde101611d0 2023-11-04T05:29:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08012b40 con 0x7fde101611d0 2023-11-04T05:29:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.807+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde100ad570 con 0x7fde101611d0 2023-11-04T05:29:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.808+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde101f6fe0 con 0x7fde101611d0 2023-11-04T05:29:30.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.809+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde08012ca0 con 0x7fde101611d0 2023-11-04T05:29:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.809+0000 7fde05ffb700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fddf406af00 0x7fddf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.809+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fde0808bc40 con 0x7fde101611d0 2023-11-04T05:29:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.809+0000 7fde1547f700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fddf406af00 0x7fddf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.810+0000 7fde1547f700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fddf406af00 0x7fddf406d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fde00007fd0 tx=0x7fde00007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:30.814+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fde101f6fe0 con 0x7fde101611d0 2023-11-04T05:29:31.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.062+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fde101f6fe0 con 0x7fde101611d0 2023-11-04T05:29:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.662+0000 7fde05ffb700 1 -- 172.21.15.3:0/2719681621 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde101f6fe0 con 0x7fde101611d0 2023-11-04T05:29:31.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fddf406af00 msgr2=0x7fddf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:31.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fddf406af00 0x7fddf406d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fde00007fd0 tx=0x7fde00007ec0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 msgr2=0x7fde101f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:31.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 0x7fde101f72c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fde100d1750 tx=0x7fde08004ad0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 shutdown_connections 2023-11-04T05:29:31.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fddf406af00 0x7fddf406d3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde101611d0 0x7fde101f72c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde1006ab20 0x7fde100c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 --2- 172.21.15.3:0/2719681621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1006afa0 0x7fde100c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 >> 172.21.15.3:0/2719681621 conn(0x7fde10072560 msgr2=0x7fde100cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:31.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 shutdown_connections 2023-11-04T05:29:31.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.665+0000 7fde176e3700 1 -- 172.21.15.3:0/2719681621 wait complete. 2023-11-04T05:29:31.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-04T05:29:31.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.816+0000 7ff2f3fff700 1 Processor -- start 2023-11-04T05:29:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.816+0000 7ff2f3fff700 1 -- start start 2023-11-04T05:29:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2f406ab20 0x7ff2f415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f3fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff2f406e970 0x7ff2f406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f3fff700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f4071cd0 con 0x7ff2f415d140 2023-11-04T05:29:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f3fff700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f41561c0 con 0x7ff2f406ab20 2023-11-04T05:29:31.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f3fff700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f406b440 con 0x7ff2f406e970 2023-11-04T05:29:31.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2f406ab20 0x7ff2f415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:31.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2f406ab20 0x7ff2f415bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53658/0 (socket says 172.21.15.3:53658) 2023-11-04T05:29:31.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f2ffd700 1 -- 172.21.15.3:0/791097655 learned_addr learned my addr 172.21.15.3:0/791097655 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f27fc700 1 --2- 172.21.15.3:0/791097655 >> v2:172.21.15.188:3300/0 conn(0x7ff2f406e970 0x7ff2f406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f2ffd700 1 -- 172.21.15.3:0/791097655 >> v2:172.21.15.188:3300/0 conn(0x7ff2f406e970 msgr2=0x7ff2f406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.817+0000 7ff2f2ffd700 1 --2- 172.21.15.3:0/791097655 >> v2:172.21.15.188:3300/0 conn(0x7ff2f406e970 0x7ff2f406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f2ffd700 1 -- 172.21.15.3:0/791097655 >> v1:172.21.15.188:6789/0 conn(0x7ff2f415d140 legacy=0x7ff2f40721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:31.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f2ffd700 1 -- 172.21.15.3:0/791097655 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f415c230 con 0x7ff2f406ab20 2023-11-04T05:29:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f2ffd700 1 --2- 172.21.15.3:0/791097655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406ab20 0x7ff2f415bc50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40719a0 tx=0x7ff2e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5089faa26324f3af server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f1ffb700 1 -- 172.21.15.3:0/791097655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800b870 con 0x7ff2f406ab20 2023-11-04T05:29:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f1ffb700 1 -- 172.21.15.3:0/791097655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e800b9d0 con 0x7ff2f406ab20 2023-11-04T05:29:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f1ffb700 1 -- 172.21.15.3:0/791097655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800bd60 con 0x7ff2f406ab20 2023-11-04T05:29:31.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 -- 172.21.15.3:0/791097655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406ab20 msgr2=0x7ff2f415bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/791097655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406ab20 0x7ff2f415bc50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40719a0 tx=0x7ff2e8009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 -- 172.21.15.3:0/791097655 shutdown_connections 2023-11-04T05:29:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/791097655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406ab20 0x7ff2f415bc50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/791097655 >> v2:172.21.15.188:3300/0 conn(0x7ff2f406e970 0x7ff2f406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 -- 172.21.15.3:0/791097655 >> 172.21.15.3:0/791097655 conn(0x7ff2f406bc30 msgr2=0x7ff2f406c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.818+0000 7ff2f3fff700 1 -- 172.21.15.3:0/791097655 shutdown_connections 2023-11-04T05:29:31.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.819+0000 7ff2f3fff700 1 -- 172.21.15.3:0/791097655 wait complete. 2023-11-04T05:29:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.820+0000 7ff2f3fff700 1 Processor -- start 2023-11-04T05:29:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.820+0000 7ff2f3fff700 1 -- start start 2023-11-04T05:29:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2f406ab20 0x7ff2f40c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 0x7ff2f4151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2f415d140 0x7ff2f41e4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40c5dd0 con 0x7ff2f406e970 2023-11-04T05:29:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40c60b0 con 0x7ff2f406ab20 2023-11-04T05:29:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40c6390 con 0x7ff2f415d140 2023-11-04T05:29:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 0x7ff2f4151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 0x7ff2f4151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53666/0 (socket says 172.21.15.3:53666) 2023-11-04T05:29:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 -- 172.21.15.3:0/4209880368 learned_addr learned my addr 172.21.15.3:0/4209880368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f2ffd700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2f406ab20 0x7ff2f40c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f37fe700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2f415d140 0x7ff2f41e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 -- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2f415d140 msgr2=0x7ff2f41e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2f415d140 0x7ff2f41e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 -- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2f406ab20 msgr2=0x7ff2f40c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2f406ab20 0x7ff2f40c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 -- 172.21.15.3:0/4209880368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009550 con 0x7ff2f406e970 2023-11-04T05:29:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f37fe700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2f415d140 0x7ff2f41e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f27fc700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 0x7ff2f4151c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400d800 tx=0x7ff2e400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40125f0 con 0x7ff2f406e970 2023-11-04T05:29:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e4012750 con 0x7ff2f406e970 2023-11-04T05:29:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f40c6670 con 0x7ff2f406e970 2023-11-04T05:29:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.821+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4012b00 con 0x7ff2f406e970 2023-11-04T05:29:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.822+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f40c5a10 con 0x7ff2f406e970 2023-11-04T05:29:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.822+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f404f730 con 0x7ff2f406e970 2023-11-04T05:29:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.823+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2e4012c60 con 0x7ff2f406e970 2023-11-04T05:29:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.823+0000 7ff2db7fe700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2e006af00 0x7ff2e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.823+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6749+0+0 (secure 0 0 0) 0x7ff2e4089e00 con 0x7ff2f406e970 2023-11-04T05:29:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.824+0000 7ff2f2ffd700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2e006af00 0x7ff2e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.824+0000 7ff2f2ffd700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2e006af00 0x7ff2e006d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8014040 tx=0x7ff2e8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:31.828+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff2e4054aa0 con 0x7ff2f406e970 2023-11-04T05:29:32.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.031+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff2f415efb0 con 0x7ff2f406e970 2023-11-04T05:29:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.669+0000 7ff2db7fe700 1 -- 172.21.15.3:0/4209880368 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff2e40580c0 con 0x7ff2f406e970 2023-11-04T05:29:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2e006af00 msgr2=0x7ff2e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:32.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2e006af00 0x7ff2e006d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8014040 tx=0x7ff2e8008030 comp rx=0 tx=0).stop 2023-11-04T05:29:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 msgr2=0x7ff2f4151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 0x7ff2f4151c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400d800 tx=0x7ff2e400dbc0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 shutdown_connections 2023-11-04T05:29:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2e006af00 0x7ff2e006d3c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2f406ab20 0x7ff2f40c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2f415d140 0x7ff2f41e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 --2- 172.21.15.3:0/4209880368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2f406e970 0x7ff2f4151c40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 >> 172.21.15.3:0/4209880368 conn(0x7ff2f406bc30 msgr2=0x7ff2f40d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:32.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 shutdown_connections 2023-11-04T05:29:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.673+0000 7ff2f3fff700 1 -- 172.21.15.3:0/4209880368 wait complete. 2023-11-04T05:29:32.677 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-11-04T05:29:32.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-04T05:29:32.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.831+0000 7f6b6a34a700 1 Processor -- start 2023-11-04T05:29:32.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.831+0000 7f6b6a34a700 1 -- start start 2023-11-04T05:29:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b6a34a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b640cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b6a34a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b6415c2d0 0x7f6b6406ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b6a34a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640b0f20 con 0x7f6b6415be50 2023-11-04T05:29:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b6a34a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6415f4a0 con 0x7f6b6415c2d0 2023-11-04T05:29:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b6a34a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64160370 con 0x7f6b64155df0 2023-11-04T05:29:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b637fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6b6415be50 0x7f6b6415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59020/0 (socket says 172.21.15.3:59020) 2023-11-04T05:29:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b637fe700 1 -- 172.21.15.3:0/2851423937 learned_addr learned my addr 172.21.15.3:0/2851423937 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b688e7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b6415c2d0 0x7f6b6406ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b63fff700 1 --2- 172.21.15.3:0/2851423937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b640cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b62ffd700 1 -- 172.21.15.3:0/2851423937 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880141239 0 0) 0x7f6b58001110 con 0x7f6b6415be50 2023-11-04T05:29:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b62ffd700 1 -- 172.21.15.3:0/2851423937 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b4c0035b0 con 0x7f6b6415be50 2023-11-04T05:29:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b688e7700 1 -- 172.21.15.3:0/2851423937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 msgr2=0x7f6b640cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b688e7700 1 --2- 172.21.15.3:0/2851423937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b640cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b688e7700 1 -- 172.21.15.3:0/2851423937 >> v1:172.21.15.3:6789/0 conn(0x7f6b6415be50 legacy=0x7f6b6415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b688e7700 1 -- 172.21.15.3:0/2851423937 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6415c770 con 0x7f6b6415c2d0 2023-11-04T05:29:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.832+0000 7f6b688e7700 1 --2- 172.21.15.3:0/2851423937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b6406ab20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009d90 tx=0x7f6b5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d40ed5d0e74b2c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b62ffd700 1 -- 172.21.15.3:0/2851423937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00e070 con 0x7f6b6415c2d0 2023-11-04T05:29:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b62ffd700 1 -- 172.21.15.3:0/2851423937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c00bb20 con 0x7f6b6415c2d0 2023-11-04T05:29:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b62ffd700 1 -- 172.21.15.3:0/2851423937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c012430 con 0x7f6b6415c2d0 2023-11-04T05:29:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 -- 172.21.15.3:0/2851423937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 msgr2=0x7f6b6406ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/2851423937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b6406ab20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009d90 tx=0x7f6b5c00b330 comp rx=0 tx=0).stop 2023-11-04T05:29:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 -- 172.21.15.3:0/2851423937 shutdown_connections 2023-11-04T05:29:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/2851423937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b6406ab20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/2851423937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b640cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 -- 172.21.15.3:0/2851423937 >> 172.21.15.3:0/2851423937 conn(0x7f6b64072560 msgr2=0x7f6b64099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 -- 172.21.15.3:0/2851423937 shutdown_connections 2023-11-04T05:29:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.833+0000 7f6b6a34a700 1 -- 172.21.15.3:0/2851423937 wait complete. 2023-11-04T05:29:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.835+0000 7f6b6a34a700 1 Processor -- start 2023-11-04T05:29:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.835+0000 7f6b6a34a700 1 -- start start 2023-11-04T05:29:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b6a34a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b64150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:32.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b6a34a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b6415be50 0x7f6b6406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b6a34a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b640b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b6a34a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6415afb0 con 0x7f6b6415c2d0 2023-11-04T05:29:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b6a34a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b641510c0 con 0x7f6b6415be50 2023-11-04T05:29:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b6a34a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6415a680 con 0x7f6b64155df0 2023-11-04T05:29:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b640b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b640b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53682/0 (socket says 172.21.15.3:53682) 2023-11-04T05:29:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 -- 172.21.15.3:0/113903866 learned_addr learned my addr 172.21.15.3:0/113903866 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b63fff700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b64150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 -- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 msgr2=0x7f6b64150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b64150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 -- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b6415be50 msgr2=0x7f6b6406f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b6415be50 0x7f6b6406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.836+0000 7f6b688e7700 1 -- 172.21.15.3:0/113903866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c009a70 con 0x7f6b6415c2d0 2023-11-04T05:29:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.837+0000 7f6b688e7700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b640b31b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c014040 tx=0x7f6b5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.837+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0129f0 con 0x7f6b6415c2d0 2023-11-04T05:29:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.837+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c020070 con 0x7f6b6415c2d0 2023-11-04T05:29:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.837+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6415a960 con 0x7f6b6415c2d0 2023-11-04T05:29:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.837+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c012d90 con 0x7f6b6415c2d0 2023-11-04T05:29:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.837+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6406fbf0 con 0x7f6b6415c2d0 2023-11-04T05:29:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.838+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6415b120 con 0x7f6b6415c2d0 2023-11-04T05:29:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.838+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b5c004040 con 0x7f6b6415c2d0 2023-11-04T05:29:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.839+0000 7f6b60ff9700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6b5006af00 0x7f6b5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.839+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f6b5c022070 con 0x7f6b6415c2d0 2023-11-04T05:29:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.839+0000 7f6b63fff700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6b5006af00 0x7f6b5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:32.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.840+0000 7f6b63fff700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6b5006af00 0x7f6b5006d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009ab0 tx=0x7f6b54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:32.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:32.843+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f6b5c05a040 con 0x7f6b6415c2d0 2023-11-04T05:29:33.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.045+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b6406ff00 con 0x7f6b6415c2d0 2023-11-04T05:29:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.830+0000 7f6b60ff9700 1 -- 172.21.15.3:0/113903866 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6b5c059bd0 con 0x7f6b6415c2d0 2023-11-04T05:29:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6b5006af00 msgr2=0x7f6b5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6b5006af00 0x7f6b5006d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009ab0 tx=0x7f6b54008040 comp rx=0 tx=0).stop 2023-11-04T05:29:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 msgr2=0x7f6b640b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b640b31b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c014040 tx=0x7f6b5c004ad0 comp rx=0 tx=0).stop 2023-11-04T05:29:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 shutdown_connections 2023-11-04T05:29:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6b5006af00 0x7f6b5006d3c0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009ab0 tx=0x7f6b54008040 comp rx=0 tx=0).stop 2023-11-04T05:29:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6415c2d0 0x7f6b640b31b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b64155df0 0x7f6b64150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 --2- 172.21.15.3:0/113903866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b6415be50 0x7f6b6406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.834+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 >> 172.21.15.3:0/113903866 conn(0x7f6b64072560 msgr2=0x7f6b6406ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.836+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 shutdown_connections 2023-11-04T05:29:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.836+0000 7f6b6a34a700 1 -- 172.21.15.3:0/113903866 wait complete. 2023-11-04T05:29:33.843 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-11-04T05:29:33.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-04T05:29:33.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.995+0000 7fa891181700 1 Processor -- start 2023-11-04T05:29:33.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.995+0000 7fa891181700 1 -- start start 2023-11-04T05:29:34.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa891181700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa88c15ba10 0x7fa88c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:34.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa891181700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa88c0d0d70 0x7fa88c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:34.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa891181700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c070a90 con 0x7fa88c0b3650 2023-11-04T05:29:34.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa891181700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c0b3ac0 con 0x7fa88c15ba10 2023-11-04T05:29:34.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa891181700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c0718f0 con 0x7fa88c0d0d70 2023-11-04T05:29:34.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa88c15ba10 0x7fa88c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:34.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa88c15ba10 0x7fa88c0c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53686/0 (socket says 172.21.15.3:53686) 2023-11-04T05:29:34.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 -- 172.21.15.3:0/3101611767 learned_addr learned my addr 172.21.15.3:0/3101611767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:34.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88a59c700 1 --2- 172.21.15.3:0/3101611767 >> v2:172.21.15.188:3300/0 conn(0x7fa88c0d0d70 0x7fa88c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:34.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 -- 172.21.15.3:0/3101611767 >> v2:172.21.15.188:3300/0 conn(0x7fa88c0d0d70 msgr2=0x7fa88c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:34.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 --2- 172.21.15.3:0/3101611767 >> v2:172.21.15.188:3300/0 conn(0x7fa88c0d0d70 0x7fa88c0b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:34.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 -- 172.21.15.3:0/3101611767 >> v1:172.21.15.188:6789/0 conn(0x7fa88c0b3650 legacy=0x7fa88c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:34.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 -- 172.21.15.3:0/3101611767 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0c5720 con 0x7fa88c15ba10 2023-11-04T05:29:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.996+0000 7fa88ad9d700 1 --2- 172.21.15.3:0/3101611767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c0c51e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0b4010 tx=0x7fa880009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eca797862d4c2f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa889d9b700 1 -- 172.21.15.3:0/3101611767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000b870 con 0x7fa88c15ba10 2023-11-04T05:29:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa889d9b700 1 -- 172.21.15.3:0/3101611767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88000b9d0 con 0x7fa88c15ba10 2023-11-04T05:29:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa889d9b700 1 -- 172.21.15.3:0/3101611767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000bd60 con 0x7fa88c15ba10 2023-11-04T05:29:34.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 -- 172.21.15.3:0/3101611767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 msgr2=0x7fa88c0c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 --2- 172.21.15.3:0/3101611767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c0c51e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0b4010 tx=0x7fa880009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 -- 172.21.15.3:0/3101611767 shutdown_connections 2023-11-04T05:29:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 --2- 172.21.15.3:0/3101611767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c0c51e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 --2- 172.21.15.3:0/3101611767 >> v2:172.21.15.188:3300/0 conn(0x7fa88c0d0d70 0x7fa88c0b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:34.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 -- 172.21.15.3:0/3101611767 >> 172.21.15.3:0/3101611767 conn(0x7fa88c0ce110 msgr2=0x7fa88c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 -- 172.21.15.3:0/3101611767 shutdown_connections 2023-11-04T05:29:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.997+0000 7fa891181700 1 -- 172.21.15.3:0/3101611767 wait complete. 2023-11-04T05:29:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.999+0000 7fa891181700 1 Processor -- start 2023-11-04T05:29:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.999+0000 7fa891181700 1 -- start start 2023-11-04T05:29:34.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.999+0000 7fa891181700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c0b3650 0x7fa88c147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.999+0000 7fa891181700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0d70 0x7fa88c0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:33.999+0000 7fa891181700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa891181700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c1484d0 con 0x7fa88c15ba10 2023-11-04T05:29:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa891181700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c15e9c0 con 0x7fa88c0b3650 2023-11-04T05:29:34.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa891181700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c15eca0 con 0x7fa88c0d0d70 2023-11-04T05:29:34.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:34.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53696/0 (socket says 172.21.15.3:53696) 2023-11-04T05:29:34.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 -- 172.21.15.3:0/3136780973 learned_addr learned my addr 172.21.15.3:0/3136780973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:34.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88ad9d700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c0b3650 0x7fa88c147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:34.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88a59c700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0d70 0x7fa88c0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 -- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0d70 msgr2=0x7fa88c0c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0d70 0x7fa88c0c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 -- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c0b3650 msgr2=0x7fa88c147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:34.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c0b3650 0x7fa88c147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 -- 172.21.15.3:0/3136780973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009550 con 0x7fa88c15ba10 2023-11-04T05:29:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa88b59e700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c15df00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa88400cc50 tx=0x7fa88400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884003c00 con 0x7fa88c15ba10 2023-11-04T05:29:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884003d60 con 0x7fa88c15ba10 2023-11-04T05:29:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400aab0 con 0x7fa88c15ba10 2023-11-04T05:29:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c147790 con 0x7fa88c15ba10 2023-11-04T05:29:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.000+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c0c48b0 con 0x7fa88c15ba10 2023-11-04T05:29:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.001+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c147cb0 con 0x7fa88c15ba10 2023-11-04T05:29:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.002+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa88400ac10 con 0x7fa88c15ba10 2023-11-04T05:29:34.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.002+0000 7fa8737fe700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa87806af00 0x7fa87806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.002+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fa8840896a0 con 0x7fa88c15ba10 2023-11-04T05:29:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.002+0000 7fa88ad9d700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa87806af00 0x7fa87806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.003+0000 7fa88ad9d700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa87806af00 0x7fa87806d3c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa880014040 tx=0x7fa880008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.007+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa88c147cb0 con 0x7fa88c15ba10 2023-11-04T05:29:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:34.207+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa88c147cb0 con 0x7fa88c15ba10 2023-11-04T05:29:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.075+0000 7fa8737fe700 1 -- 172.21.15.3:0/3136780973 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7fa88c147cb0 con 0x7fa88c15ba10 2023-11-04T05:29:35.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa87806af00 msgr2=0x7fa87806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:35.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa87806af00 0x7fa87806d3c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa880014040 tx=0x7fa880008030 comp rx=0 tx=0).stop 2023-11-04T05:29:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 msgr2=0x7fa88c15df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c15df00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa88400cc50 tx=0x7fa88400cf60 comp rx=0 tx=0).stop 2023-11-04T05:29:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 shutdown_connections 2023-11-04T05:29:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa87806af00 0x7fa87806d3c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c0b3650 0x7fa88c147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa88c15ba10 0x7fa88c15df00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 --2- 172.21.15.3:0/3136780973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0d70 0x7fa88c0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 >> 172.21.15.3:0/3136780973 conn(0x7fa88c0ce110 msgr2=0x7fa88c0cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:35.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 shutdown_connections 2023-11-04T05:29:35.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.078+0000 7fa891181700 1 -- 172.21.15.3:0/3136780973 wait complete. 2023-11-04T05:29:35.083 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' removed 2023-11-04T05:29:35.099 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-04T05:29:35.099 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-04T05:29:35.100 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-04T05:29:35.100 INFO:teuthology.orchestra.run:waiting for 300 2023-11-04T05:29:35.100 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-04T05:29:35.100 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-04T05:29:35.101 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-04T05:29:35.101 INFO:teuthology.orchestra.run:waiting for 300 2023-11-04T05:29:35.101 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-04T05:29:35.102 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-04T05:29:35.102 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-04T05:29:35.102 INFO:teuthology.orchestra.run:waiting for 300 2023-11-04T05:29:35.103 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-04T05:29:35.103 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-04T05:29:35.103 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-04T05:29:35.103 INFO:teuthology.orchestra.run:waiting for 300 2023-11-04T05:29:35.109 INFO:tasks.ceph.mds.a:Stopped 2023-11-04T05:29:35.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-04T05:29:35.111 INFO:tasks.ceph.mds.c:Stopped 2023-11-04T05:29:35.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-04T05:29:35.113 INFO:tasks.ceph.mds.b:Stopped 2023-11-04T05:29:35.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-04T05:29:35.114 INFO:tasks.ceph.mds.d:Stopped 2023-11-04T05:29:35.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-04T05:29:35.152 INFO:tasks.ceph.mds.a:Started 2023-11-04T05:29:35.153 DEBUG:teuthology.parallel:result is None 2023-11-04T05:29:35.154 INFO:tasks.ceph.mds.c:Started 2023-11-04T05:29:35.155 DEBUG:teuthology.parallel:result is None 2023-11-04T05:29:35.157 INFO:tasks.ceph.mds.b:Started 2023-11-04T05:29:35.157 DEBUG:teuthology.parallel:result is None 2023-11-04T05:29:35.158 INFO:tasks.ceph.mds.d:Started 2023-11-04T05:29:35.158 DEBUG:teuthology.parallel:result is None 2023-11-04T05:29:35.159 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-04T05:29:35.240 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-04T05:29:35.240 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-04T05:29:35.244 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-11-04T05:29:35.245 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-11-04T05:29:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.298+0000 7f7ad131e700 1 Processor -- start 2023-11-04T05:29:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.298+0000 7f7ad131e700 1 -- start start 2023-11-04T05:29:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7ad131e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7ad131e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7acc066970 0x7f7acc06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7ad131e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7acc06c070 0x7f7acc0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7ad131e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc06c500 con 0x7f7acc066970 2023-11-04T05:29:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7ad131e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc06d570 con 0x7f7acc06c070 2023-11-04T05:29:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7ad131e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0cf630 con 0x7f7acc15ac90 2023-11-04T05:29:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7aca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7acc066970 0x7f7acc06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7aca7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7acc066970 0x7f7acc06bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53724/0 (socket says 172.21.15.3:53724) 2023-11-04T05:29:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.299+0000 7f7aca7fc700 1 -- 172.21.15.3:0/3426145485 learned_addr learned my addr 172.21.15.3:0/3426145485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7acb7fe700 1 --2- 172.21.15.3:0/3426145485 >> v2:172.21.15.188:3300/0 conn(0x7f7acc06c070 0x7f7acc0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7acaffd700 1 --2- 172.21.15.3:0/3426145485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7aca7fc700 1 -- 172.21.15.3:0/3426145485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 msgr2=0x7f7acc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7aca7fc700 1 --2- 172.21.15.3:0/3426145485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7aca7fc700 1 -- 172.21.15.3:0/3426145485 >> v2:172.21.15.188:3300/0 conn(0x7f7acc06c070 msgr2=0x7f7acc0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7aca7fc700 1 --2- 172.21.15.3:0/3426145485 >> v2:172.21.15.188:3300/0 conn(0x7f7acc06c070 0x7f7acc0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7aca7fc700 1 -- 172.21.15.3:0/3426145485 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc06d810 con 0x7f7acc066970 2023-11-04T05:29:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7acb7fe700 1 --2- 172.21.15.3:0/3426145485 >> v2:172.21.15.188:3300/0 conn(0x7f7acc06c070 0x7f7acc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:35.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7aca7fc700 1 --2- 172.21.15.3:0/3426145485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc066970 0x7f7acc06bb30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7acc06cf40 tx=0x7f7ac000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7de31b8d9afdadd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7ac9ffb700 1 -- 172.21.15.3:0/3426145485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000dd30 con 0x7f7acc066970 2023-11-04T05:29:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7ac9ffb700 1 -- 172.21.15.3:0/3426145485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000de90 con 0x7f7acc066970 2023-11-04T05:29:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7ac9ffb700 1 -- 172.21.15.3:0/3426145485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0010690 con 0x7f7acc066970 2023-11-04T05:29:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.300+0000 7f7ad131e700 1 -- 172.21.15.3:0/3426145485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc066970 msgr2=0x7f7acc06bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:35.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 --2- 172.21.15.3:0/3426145485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc066970 0x7f7acc06bb30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7acc06cf40 tx=0x7f7ac000be70 comp rx=0 tx=0).stop 2023-11-04T05:29:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 -- 172.21.15.3:0/3426145485 shutdown_connections 2023-11-04T05:29:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 --2- 172.21.15.3:0/3426145485 >> v2:172.21.15.188:3300/0 conn(0x7f7acc06c070 0x7f7acc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 --2- 172.21.15.3:0/3426145485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 --2- 172.21.15.3:0/3426145485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc066970 0x7f7acc06bb30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 -- 172.21.15.3:0/3426145485 >> 172.21.15.3:0/3426145485 conn(0x7f7acc0716c0 msgr2=0x7f7acc071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 -- 172.21.15.3:0/3426145485 shutdown_connections 2023-11-04T05:29:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.301+0000 7f7ad131e700 1 -- 172.21.15.3:0/3426145485 wait complete. 2023-11-04T05:29:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.302+0000 7f7ad131e700 1 Processor -- start 2023-11-04T05:29:35.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.302+0000 7f7ad131e700 1 -- start start 2023-11-04T05:29:35.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7acc066970 0x7f7acc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 0x7f7acc0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc1ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0c8050 con 0x7f7acc06c070 2023-11-04T05:29:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0c8470 con 0x7f7acc066970 2023-11-04T05:29:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0c73d0 con 0x7f7acc15ac90 2023-11-04T05:29:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 0x7f7acc0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 0x7f7acc0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53732/0 (socket says 172.21.15.3:53732) 2023-11-04T05:29:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 -- 172.21.15.3:0/4275206649 learned_addr learned my addr 172.21.15.3:0/4275206649 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7acaffd700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7acc066970 0x7f7acc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7acb7fe700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc1ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 -- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 msgr2=0x7f7acc1ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc1ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 -- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7acc066970 msgr2=0x7f7acc0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7acc066970 0x7f7acc0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 -- 172.21.15.3:0/4275206649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac000b7f0 con 0x7f7acc06c070 2023-11-04T05:29:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7acb7fe700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc1ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7aca7fc700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 0x7f7acc0b43b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000beb0 tx=0x7f7ac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac001f070 con 0x7f7acc06c070 2023-11-04T05:29:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac0003bd0 con 0x7f7acc06c070 2023-11-04T05:29:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc0c76b0 con 0x7f7acc06c070 2023-11-04T05:29:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.303+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0010470 con 0x7f7acc06c070 2023-11-04T05:29:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.304+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc0c7c70 con 0x7f7acc06c070 2023-11-04T05:29:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.304+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc0ca8e0 con 0x7f7acc06c070 2023-11-04T05:29:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.305+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ac0019400 con 0x7f7acc06c070 2023-11-04T05:29:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.305+0000 7f7ab37fe700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7ab806af00 0x7f7ab806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.306+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7ac008c940 con 0x7f7acc06c070 2023-11-04T05:29:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.306+0000 7f7acaffd700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7ab806af00 0x7f7ab806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.306+0000 7f7acaffd700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7ab806af00 0x7f7ab806d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40070b0 tx=0x7f7ac4007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.310+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7ac0057c40 con 0x7f7acc06c070 2023-11-04T05:29:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:35.527+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7acc0c8190 con 0x7f7acc06c070 2023-11-04T05:29:36.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.088+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4275206649 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7acc0c8190 con 0x7f7acc06c070 2023-11-04T05:29:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7ab806af00 msgr2=0x7f7ab806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7ab806af00 0x7f7ab806d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40070b0 tx=0x7f7ac4007040 comp rx=0 tx=0).stop 2023-11-04T05:29:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 msgr2=0x7f7acc0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 0x7f7acc0b43b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000beb0 tx=0x7f7ac0004750 comp rx=0 tx=0).stop 2023-11-04T05:29:36.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 shutdown_connections 2023-11-04T05:29:36.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7ab806af00 0x7f7ab806d3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7acc066970 0x7f7acc0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7acc15ac90 0x7f7acc1ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 --2- 172.21.15.3:0/4275206649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7acc06c070 0x7f7acc0b43b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.090+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 >> 172.21.15.3:0/4275206649 conn(0x7f7acc0716c0 msgr2=0x7f7acc156890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:36.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.091+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 shutdown_connections 2023-11-04T05:29:36.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.091+0000 7f7ad131e700 1 -- 172.21.15.3:0/4275206649 wait complete. 2023-11-04T05:29:36.096 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-11-04T05:29:36.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-04T05:29:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.241+0000 7fed817a6700 1 Processor -- start 2023-11-04T05:29:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.241+0000 7fed817a6700 1 -- start start 2023-11-04T05:29:36.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed817a6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed7c1612c0 0x7fed7c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed817a6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed7c0d2690 0x7fed7c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed817a6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c1616a0 con 0x7fed7c0b2e00 2023-11-04T05:29:36.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed817a6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0b2bb0 con 0x7fed7c0d2690 2023-11-04T05:29:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed817a6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0d42c0 con 0x7fed7c1612c0 2023-11-04T05:29:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed7c0d2690 0x7fed7c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed7c0d2690 0x7fed7c0d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53742/0 (socket says 172.21.15.3:53742) 2023-11-04T05:29:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 -- 172.21.15.3:0/4219036709 learned_addr learned my addr 172.21.15.3:0/4219036709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7affd700 1 --2- 172.21.15.3:0/4219036709 >> v2:172.21.15.188:3300/0 conn(0x7fed7c1612c0 0x7fed7c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 -- 172.21.15.3:0/4219036709 >> v2:172.21.15.188:3300/0 conn(0x7fed7c1612c0 msgr2=0x7fed7c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 --2- 172.21.15.3:0/4219036709 >> v2:172.21.15.188:3300/0 conn(0x7fed7c1612c0 0x7fed7c06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 -- 172.21.15.3:0/4219036709 >> v1:172.21.15.188:6789/0 conn(0x7fed7c0b2e00 legacy=0x7fed7c0d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 -- 172.21.15.3:0/4219036709 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0d45a0 con 0x7fed7c0d2690 2023-11-04T05:29:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7affd700 1 --2- 172.21.15.3:0/4219036709 >> v2:172.21.15.188:3300/0 conn(0x7fed7c1612c0 0x7fed7c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.242+0000 7fed7b7fe700 1 --2- 172.21.15.3:0/4219036709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c0d2690 0x7fed7c0d1780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed7c08e0e0 tx=0x7fed7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a755581d8ac549dc server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed79ffb700 1 -- 172.21.15.3:0/4219036709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7400e070 con 0x7fed7c0d2690 2023-11-04T05:29:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed79ffb700 1 -- 172.21.15.3:0/4219036709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed74003a20 con 0x7fed7c0d2690 2023-11-04T05:29:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed79ffb700 1 -- 172.21.15.3:0/4219036709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74003dd0 con 0x7fed7c0d2690 2023-11-04T05:29:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 -- 172.21.15.3:0/4219036709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c0d2690 msgr2=0x7fed7c0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 --2- 172.21.15.3:0/4219036709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c0d2690 0x7fed7c0d1780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed7c08e0e0 tx=0x7fed7400b330 comp rx=0 tx=0).stop 2023-11-04T05:29:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 -- 172.21.15.3:0/4219036709 shutdown_connections 2023-11-04T05:29:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 --2- 172.21.15.3:0/4219036709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c0d2690 0x7fed7c0d1780 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 --2- 172.21.15.3:0/4219036709 >> v2:172.21.15.188:3300/0 conn(0x7fed7c1612c0 0x7fed7c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 -- 172.21.15.3:0/4219036709 >> 172.21.15.3:0/4219036709 conn(0x7fed7c08aff0 msgr2=0x7fed7c0d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 -- 172.21.15.3:0/4219036709 shutdown_connections 2023-11-04T05:29:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.243+0000 7fed817a6700 1 -- 172.21.15.3:0/4219036709 wait complete. 2023-11-04T05:29:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 Processor -- start 2023-11-04T05:29:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 -- start start 2023-11-04T05:29:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed7c0b2e00 0x7fed7c1f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed7c0d2690 0x7fed7c1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 0x7fed7c1f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c14a8d0 con 0x7fed7c1612c0 2023-11-04T05:29:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c149e20 con 0x7fed7c0d2690 2023-11-04T05:29:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.245+0000 7fed817a6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0cbcf0 con 0x7fed7c0b2e00 2023-11-04T05:29:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 0x7fed7c1f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 0x7fed7c1f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53748/0 (socket says 172.21.15.3:53748) 2023-11-04T05:29:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 -- 172.21.15.3:0/1218440113 learned_addr learned my addr 172.21.15.3:0/1218440113 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7a7fc700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed7c0d2690 0x7fed7c1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7affd700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed7c0b2e00 0x7fed7c1f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 -- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed7c0b2e00 msgr2=0x7fed7c1f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed7c0b2e00 0x7fed7c1f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 -- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed7c0d2690 msgr2=0x7fed7c1498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed7c0d2690 0x7fed7c1498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 -- 172.21.15.3:0/1218440113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74009b30 con 0x7fed7c1612c0 2023-11-04T05:29:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed7b7fe700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 0x7fed7c1f2e30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed74015040 tx=0x7fed74004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74004440 con 0x7fed7c1612c0 2023-11-04T05:29:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed740045a0 con 0x7fed7c1612c0 2023-11-04T05:29:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c14a030 con 0x7fed7c1612c0 2023-11-04T05:29:36.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74004950 con 0x7fed7c1612c0 2023-11-04T05:29:36.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.246+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c14a5f0 con 0x7fed7c1612c0 2023-11-04T05:29:36.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.247+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c0cbac0 con 0x7fed7c1612c0 2023-11-04T05:29:36.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.248+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed74012430 con 0x7fed7c1612c0 2023-11-04T05:29:36.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.248+0000 7fed637fe700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fed6806af00 0x7fed6806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.248+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fed7408a870 con 0x7fed7c1612c0 2023-11-04T05:29:36.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.248+0000 7fed7affd700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fed6806af00 0x7fed6806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.249+0000 7fed7affd700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fed6806af00 0x7fed6806d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b840 tx=0x7fed70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:36.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.252+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fed7c0cbac0 con 0x7fed7c1612c0 2023-11-04T05:29:36.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.538+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fed7c0cbac0 con 0x7fed7c1612c0 2023-11-04T05:29:36.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.539+0000 7fed637fe700 1 -- 172.21.15.3:0/1218440113 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fed7c0cbac0 con 0x7fed7c1612c0 2023-11-04T05:29:36.541 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:36.541 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-11-04T05:29:36.542 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-11-04T05:29:36.542 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.542 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-11-04T05:29:36.542 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA71kVlniCOOxAAiaVg5Rpzoj155dhK4fwAHQ==", 2023-11-04T05:29:36.542 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.543 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.543 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-11-04T05:29:36.543 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-11-04T05:29:36.543 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.543 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-11-04T05:29:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVlaAdTDBAADA5oMR/5/9YUnTrpYeanLg==", 2023-11-04T05:29:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-11-04T05:29:36.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-11-04T05:29:36.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.546 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.546 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.546 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-11-04T05:29:36.546 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVlmtFXBxAAOAFVrP5GVw0bydAISIXwWg==", 2023-11-04T05:29:36.547 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-11-04T05:29:36.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-11-04T05:29:36.547 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.548 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.548 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.548 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-11-04T05:29:36.548 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVlzqCbExAAIiPfbDlu1QpBQoFrNWdX9w==", 2023-11-04T05:29:36.548 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-11-04T05:29:36.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-11-04T05:29:36.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.550 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-11-04T05:29:36.550 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/1kVlv1gBORAAc9MfN39eZhn0Ev0RqWmWNw==", 2023-11-04T05:29:36.550 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.551 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.551 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.551 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.552 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.552 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-11-04T05:29:36.552 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBC1kVlhjW2OBAATJb/Xgl1X54Q0e+dYvDaSg==", 2023-11-04T05:29:36.552 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.552 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.552 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.553 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.553 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.553 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.553 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-11-04T05:29:36.554 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBF1kVliJUcOhAADbEH/8dZbGmzyxRzGb+JGw==", 2023-11-04T05:29:36.554 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.554 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.554 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.554 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.555 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.555 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.555 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-11-04T05:29:36.555 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBI1kVlLb3TORAAbSvfgbvXCFbkEFqy8hSfug==", 2023-11-04T05:29:36.556 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.556 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.556 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.557 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.557 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.558 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-11-04T05:29:36.558 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBP1kVl0Y8LAxAAvfeWJ5QbDy+rzQtCXQ3BnA==", 2023-11-04T05:29:36.558 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.558 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.559 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.559 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.559 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.559 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.559 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-11-04T05:29:36.560 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBS1kVlkbDDAhAA73MmMy6gIC2QbBqQd+HEBw==", 2023-11-04T05:29:36.560 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.560 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.560 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.560 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.561 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.561 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.561 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-11-04T05:29:36.561 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBV1kVl6XVLAhAAGIbTe6PGdKGQaqFwRFFM2A==", 2023-11-04T05:29:36.562 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.562 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.562 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.562 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.562 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.563 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.563 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-11-04T05:29:36.563 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBY1kVlxDCmARAA1mWAqF85oruCXXWnUWov7Q==", 2023-11-04T05:29:36.563 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.564 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-11-04T05:29:36.564 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-11-04T05:29:36.564 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.564 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.564 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.565 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-11-04T05:29:36.565 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVl1BIWGBAAag9+v589VkQNzEcgULo1sw==", 2023-11-04T05:29:36.565 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.565 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.565 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-11-04T05:29:36.566 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-11-04T05:29:36.566 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-11-04T05:29:36.566 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.567 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.567 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-11-04T05:29:36.567 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVl2LH1HhAAbFY/rBalHt4ukIO4XAtcKg==", 2023-11-04T05:29:36.567 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.568 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.568 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-11-04T05:29:36.568 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-11-04T05:29:36.568 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-11-04T05:29:36.568 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.569 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.569 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-11-04T05:29:36.569 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVlO86BGxAAqqXGqY9EIqyE+vWna1JW4A==", 2023-11-04T05:29:36.569 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.569 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.569 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-11-04T05:29:36.570 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-11-04T05:29:36.570 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-11-04T05:29:36.570 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.570 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.571 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-11-04T05:29:36.571 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA81kVljlRMIhAAsld5Ip1gYgLtovVaY1oRzQ==", 2023-11-04T05:29:36.571 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.571 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-11-04T05:29:36.571 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-11-04T05:29:36.572 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-11-04T05:29:36.572 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-11-04T05:29:36.572 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.572 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.572 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-11-04T05:29:36.573 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg==", 2023-11-04T05:29:36.573 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.573 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-11-04T05:29:36.573 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-11-04T05:29:36.573 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-11-04T05:29:36.573 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.574 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.574 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.574 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2023-11-04T05:29:36.574 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBe1kVleHW8AhAAVRbzALpdstoXzOZgPFeVyA==", 2023-11-04T05:29:36.575 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.575 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2023-11-04T05:29:36.575 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.575 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.576 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2023-11-04T05:29:36.576 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBe1kVl+H+8AhAAxY4l9IaMUqZ3br/DINrlfg==", 2023-11-04T05:29:36.576 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.576 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-04T05:29:36.576 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.577 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.577 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2023-11-04T05:29:36.577 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBe1kVlLYq8AhAAHzvBaNgXUfjXPBkXNuoWfw==", 2023-11-04T05:29:36.577 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.577 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2023-11-04T05:29:36.578 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.578 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.578 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2023-11-04T05:29:36.578 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBe1kVlWpK8AhAAuQWfTX0zly1r0cm46u+0qg==", 2023-11-04T05:29:36.579 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.579 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-04T05:29:36.579 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.579 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.580 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-04T05:29:36.580 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBe1kVlqJq8AhAAIeexiWTDfFrxgiVwlPWUvQ==", 2023-11-04T05:29:36.580 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.580 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-04T05:29:36.580 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.581 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.581 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2023-11-04T05:29:36.581 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBe1kVlH6O8AhAAr5SKxTWXVFMdGUGQ5Kx3Qw==", 2023-11-04T05:29:36.581 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.581 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-04T05:29:36.582 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.582 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.582 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-11-04T05:29:36.582 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA71kVlk1EQOBAAJ26wOo3iW90PE7axk7rbCQ==", 2023-11-04T05:29:36.583 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.583 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-11-04T05:29:36.583 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-11-04T05:29:36.583 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.583 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-11-04T05:29:36.584 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:36.584 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-11-04T05:29:36.584 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA71kVlYrMKNBAAFbYxO8ZiJ0xCCE1QYCt5RA==", 2023-11-04T05:29:36.584 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-11-04T05:29:36.584 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-11-04T05:29:36.585 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-11-04T05:29:36.585 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-11-04T05:29:36.585 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.585 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:36.586 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-11-04T05:29:36.586 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-11-04T05:29:36.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fed6806af00 msgr2=0x7fed6806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fed6806af00 0x7fed6806d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b840 tx=0x7fed70008040 comp rx=0 tx=0).stop 2023-11-04T05:29:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 msgr2=0x7fed7c1f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 0x7fed7c1f2e30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed74015040 tx=0x7fed74004ad0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 shutdown_connections 2023-11-04T05:29:36.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fed6806af00 0x7fed6806d3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed7c1612c0 0x7fed7c1f2e30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed7c0b2e00 0x7fed7c1f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 --2- 172.21.15.3:0/1218440113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed7c0d2690 0x7fed7c1498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 >> 172.21.15.3:0/1218440113 conn(0x7fed7c08aff0 msgr2=0x7fed7c0b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:36.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 shutdown_connections 2023-11-04T05:29:36.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.541+0000 7fed817a6700 1 -- 172.21.15.3:0/1218440113 wait complete. 2023-11-04T05:29:36.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-04T05:29:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.714+0000 7f7dfa3f7700 1 Processor -- start 2023-11-04T05:29:36.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.714+0000 7f7dfa3f7700 1 -- start start 2023-11-04T05:29:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7dfa3f7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df406ad90 0x7f7df40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7dfa3f7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7df40cdb30 0x7f7df40cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7dfa3f7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df40b1ce0 con 0x7f7df40ce130 2023-11-04T05:29:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7dfa3f7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df40ccd70 con 0x7f7df406ad90 2023-11-04T05:29:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7dfa3f7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df40cd050 con 0x7f7df40cdb30 2023-11-04T05:29:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df406ad90 0x7f7df40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df8994700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7df40ce130 0x7f7df40cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59104/0 (socket says 172.21.15.3:59104) 2023-11-04T05:29:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df8994700 1 -- 172.21.15.3:0/3011242089 learned_addr learned my addr 172.21.15.3:0/3011242089 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df37fe700 1 --2- 172.21.15.3:0/3011242089 >> v2:172.21.15.188:3300/0 conn(0x7f7df40cdb30 0x7f7df40cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df3fff700 1 -- 172.21.15.3:0/3011242089 >> v2:172.21.15.188:3300/0 conn(0x7f7df40cdb30 msgr2=0x7f7df40cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:36.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df3fff700 1 --2- 172.21.15.3:0/3011242089 >> v2:172.21.15.188:3300/0 conn(0x7f7df40cdb30 0x7f7df40cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df3fff700 1 -- 172.21.15.3:0/3011242089 >> v1:172.21.15.3:6789/0 conn(0x7f7df40ce130 legacy=0x7f7df40cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.715+0000 7f7df3fff700 1 -- 172.21.15.3:0/3011242089 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df40cd330 con 0x7f7df406ad90 2023-11-04T05:29:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7df3fff700 1 --2- 172.21.15.3:0/3011242089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df40b1490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7df40ce080 tx=0x7f7ddc009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8c6981ae977a59 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7df2ffd700 1 -- 172.21.15.3:0/3011242089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc005e30 con 0x7f7df406ad90 2023-11-04T05:29:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7df2ffd700 1 -- 172.21.15.3:0/3011242089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc006030 con 0x7f7df406ad90 2023-11-04T05:29:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7df2ffd700 1 -- 172.21.15.3:0/3011242089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0063c0 con 0x7f7df406ad90 2023-11-04T05:29:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/3011242089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 msgr2=0x7f7df40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/3011242089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df40b1490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7df40ce080 tx=0x7f7ddc009b00 comp rx=0 tx=0).stop 2023-11-04T05:29:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/3011242089 shutdown_connections 2023-11-04T05:29:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/3011242089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df40b1490 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/3011242089 >> v2:172.21.15.188:3300/0 conn(0x7f7df40cdb30 0x7f7df40cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/3011242089 >> 172.21.15.3:0/3011242089 conn(0x7f7df40d0560 msgr2=0x7f7df40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/3011242089 shutdown_connections 2023-11-04T05:29:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.716+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/3011242089 wait complete. 2023-11-04T05:29:36.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.718+0000 7f7dfa3f7700 1 Processor -- start 2023-11-04T05:29:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.718+0000 7f7dfa3f7700 1 -- start start 2023-11-04T05:29:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.718+0000 7f7dfa3f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df41f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40cdb30 0x7f7df4147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7df40ce130 0x7f7df41472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df4159ef0 con 0x7f7df406ad90 2023-11-04T05:29:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41f6f50 con 0x7f7df40cdb30 2023-11-04T05:29:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41f7230 con 0x7f7df40ce130 2023-11-04T05:29:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df41f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df41f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53758/0 (socket says 172.21.15.3:53758) 2023-11-04T05:29:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 -- 172.21.15.3:0/1190632329 learned_addr learned my addr 172.21.15.3:0/1190632329 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df37fe700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40cdb30 0x7f7df4147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df8994700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7df40ce130 0x7f7df41472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 -- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7df40ce130 msgr2=0x7f7df41472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7df40ce130 0x7f7df41472b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 -- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40cdb30 msgr2=0x7f7df4147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:36.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40cdb30 0x7f7df4147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:36.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 -- 172.21.15.3:0/1190632329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc009430 con 0x7f7df406ad90 2023-11-04T05:29:36.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df3fff700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df41f6970 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009fd0 tx=0x7f7ddc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:36.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc004040 con 0x7f7df406ad90 2023-11-04T05:29:36.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc020070 con 0x7f7df406ad90 2023-11-04T05:29:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df41f7510 con 0x7f7df406ad90 2023-11-04T05:29:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc004400 con 0x7f7df406ad90 2023-11-04T05:29:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.719+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df41f77f0 con 0x7f7df406ad90 2023-11-04T05:29:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.720+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df4159d30 con 0x7f7df406ad90 2023-11-04T05:29:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.721+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ddc022030 con 0x7f7df406ad90 2023-11-04T05:29:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.721+0000 7f7df0ff9700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7dd406af00 0x7f7dd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.721+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7ddc08a780 con 0x7f7df406ad90 2023-11-04T05:29:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.721+0000 7f7df37fe700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7dd406af00 0x7f7dd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:36.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.722+0000 7f7df37fe700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7dd406af00 0x7f7dd406d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7df40b0a00 tx=0x7f7de4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:36.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:36.726+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7df4159d30 con 0x7f7df406ad90 2023-11-04T05:29:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.009+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7df4159d30 con 0x7f7df406ad90 2023-11-04T05:29:37.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.011+0000 7f7df0ff9700 1 -- 172.21.15.3:0/1190632329 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7df4159d30 con 0x7f7df406ad90 2023-11-04T05:29:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7dd406af00 msgr2=0x7f7dd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7dd406af00 0x7f7dd406d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7df40b0a00 tx=0x7f7de4008040 comp rx=0 tx=0).stop 2023-11-04T05:29:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 msgr2=0x7f7df41f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df41f6970 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009fd0 tx=0x7f7ddc004750 comp rx=0 tx=0).stop 2023-11-04T05:29:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 shutdown_connections 2023-11-04T05:29:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f7dd406af00 0x7f7dd406d3c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7df40ce130 0x7f7df41472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df406ad90 0x7f7df41f6970 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 --2- 172.21.15.3:0/1190632329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40cdb30 0x7f7df4147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 >> 172.21.15.3:0/1190632329 conn(0x7f7df40d0560 msgr2=0x7f7df4160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 shutdown_connections 2023-11-04T05:29:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.014+0000 7f7dfa3f7700 1 -- 172.21.15.3:0/1190632329 wait complete. 2023-11-04T05:29:37.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-04T05:29:37.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 Processor -- start 2023-11-04T05:29:37.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 -- start start 2023-11-04T05:29:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91440cdb30 0x7f914415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91440ce130 0x7f914415d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440b1ce0 con 0x7f914406ad90 2023-11-04T05:29:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91440ccd70 con 0x7f91440cdb30 2023-11-04T05:29:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.154+0000 7f9149157700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91440cd050 con 0x7f91440ce130 2023-11-04T05:29:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91440cdb30 0x7f914415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91440cdb30 0x7f914415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53772/0 (socket says 172.21.15.3:53772) 2023-11-04T05:29:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 -- 172.21.15.3:0/3102970649 learned_addr learned my addr 172.21.15.3:0/3102970649 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914359e700 1 --2- 172.21.15.3:0/3102970649 >> v2:172.21.15.188:3300/0 conn(0x7f91440ce130 0x7f914415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 -- 172.21.15.3:0/3102970649 >> v2:172.21.15.188:3300/0 conn(0x7f91440ce130 msgr2=0x7f914415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:37.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 --2- 172.21.15.3:0/3102970649 >> v2:172.21.15.188:3300/0 conn(0x7f91440ce130 0x7f914415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 -- 172.21.15.3:0/3102970649 >> v1:172.21.15.188:6789/0 conn(0x7f914406ad90 legacy=0x7f91440b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 -- 172.21.15.3:0/3102970649 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440cd330 con 0x7f91440cdb30 2023-11-04T05:29:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f914259c700 1 --2- 172.21.15.3:0/3102970649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cdb30 0x7f914415d1e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9144072f60 tx=0x7f9138009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea31a06f17ae1f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f9141d9b700 1 -- 172.21.15.3:0/3102970649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800b870 con 0x7f91440cdb30 2023-11-04T05:29:37.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f9141d9b700 1 -- 172.21.15.3:0/3102970649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800b9d0 con 0x7f91440cdb30 2023-11-04T05:29:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.155+0000 7f9141d9b700 1 -- 172.21.15.3:0/3102970649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800bd60 con 0x7f91440cdb30 2023-11-04T05:29:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 -- 172.21.15.3:0/3102970649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cdb30 msgr2=0x7f914415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 --2- 172.21.15.3:0/3102970649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cdb30 0x7f914415d1e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9144072f60 tx=0x7f9138009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 -- 172.21.15.3:0/3102970649 shutdown_connections 2023-11-04T05:29:37.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 --2- 172.21.15.3:0/3102970649 >> v2:172.21.15.188:3300/0 conn(0x7f91440ce130 0x7f914415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 --2- 172.21.15.3:0/3102970649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cdb30 0x7f914415d1e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 -- 172.21.15.3:0/3102970649 >> 172.21.15.3:0/3102970649 conn(0x7f91440d0560 msgr2=0x7f91440d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 -- 172.21.15.3:0/3102970649 shutdown_connections 2023-11-04T05:29:37.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.156+0000 7f9149157700 1 -- 172.21.15.3:0/3102970649 wait complete. 2023-11-04T05:29:37.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.157+0000 7f9149157700 1 Processor -- start 2023-11-04T05:29:37.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.157+0000 7f9149157700 1 -- start start 2023-11-04T05:29:37.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9149157700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 0x7f91440c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9149157700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91440cdb30 0x7f914415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9149157700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91440ce130 0x7f91440b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9149157700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c8700 con 0x7f914406ad90 2023-11-04T05:29:37.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9149157700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c5a90 con 0x7f91440cdb30 2023-11-04T05:29:37.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9149157700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c4f90 con 0x7f91440ce130 2023-11-04T05:29:37.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9142d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 0x7f91440c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9142d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 0x7f91440c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53776/0 (socket says 172.21.15.3:53776) 2023-11-04T05:29:37.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f9142d9d700 1 -- 172.21.15.3:0/3670919606 learned_addr learned my addr 172.21.15.3:0/3670919606 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:37.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f914259c700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91440cdb30 0x7f914415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.158+0000 7f914359e700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91440ce130 0x7f91440b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9142d9d700 1 -- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91440ce130 msgr2=0x7f91440b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9142d9d700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91440ce130 0x7f91440b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9142d9d700 1 -- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91440cdb30 msgr2=0x7f914415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9142d9d700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91440cdb30 0x7f914415ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9142d9d700 1 -- 172.21.15.3:0/3670919606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009550 con 0x7f914406ad90 2023-11-04T05:29:37.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9142d9d700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 0x7f91440c4a50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f913400c4d0 tx=0x7f913400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:37.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91340052d0 con 0x7f914406ad90 2023-11-04T05:29:37.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9134005430 con 0x7f914406ad90 2023-11-04T05:29:37.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440c51f0 con 0x7f914406ad90 2023-11-04T05:29:37.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91340038f0 con 0x7f914406ad90 2023-11-04T05:29:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.159+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440c57b0 con 0x7f914406ad90 2023-11-04T05:29:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.160+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9144074df0 con 0x7f914406ad90 2023-11-04T05:29:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.161+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9134003a50 con 0x7f914406ad90 2023-11-04T05:29:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.161+0000 7f912b7fe700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f913006af00 0x7f913006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.161+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9134088830 con 0x7f914406ad90 2023-11-04T05:29:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.161+0000 7f914259c700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f913006af00 0x7f913006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.162+0000 7f914259c700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f913006af00 0x7f913006d3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9144072f60 tx=0x7f9138008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.165+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f9134053af0 con 0x7f914406ad90 2023-11-04T05:29:37.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.448+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f91440af850 con 0x7f914406ad90 2023-11-04T05:29:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.450+0000 7f912b7fe700 1 -- 172.21.15.3:0/3670919606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f91340170a0 con 0x7f914406ad90 2023-11-04T05:29:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f913006af00 msgr2=0x7f913006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f913006af00 0x7f913006d3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9144072f60 tx=0x7f9138008030 comp rx=0 tx=0).stop 2023-11-04T05:29:37.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 msgr2=0x7f91440c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 0x7f91440c4a50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f913400c4d0 tx=0x7f913400c7e0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 shutdown_connections 2023-11-04T05:29:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f913006af00 0x7f913006d3c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91440ce130 0x7f91440b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914406ad90 0x7f91440c4a50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 --2- 172.21.15.3:0/3670919606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91440cdb30 0x7f914415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 >> 172.21.15.3:0/3670919606 conn(0x7f91440d0560 msgr2=0x7f91440b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:37.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 shutdown_connections 2023-11-04T05:29:37.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.452+0000 7f9149157700 1 -- 172.21.15.3:0/3670919606 wait complete. 2023-11-04T05:29:37.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-04T05:29:37.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.590+0000 7f9f483ea700 1 Processor -- start 2023-11-04T05:29:37.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.590+0000 7f9f483ea700 1 -- start start 2023-11-04T05:29:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.590+0000 7f9f483ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 0x7f9f400b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f483ea700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f400cb9d0 con 0x7f9f400ce130 2023-11-04T05:29:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f483ea700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f400b1ce0 con 0x7f9f400cdb30 2023-11-04T05:29:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f483ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400ccd70 con 0x7f9f4006ad90 2023-11-04T05:29:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f46987700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9f400ce130 0x7f9f400cb2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59136/0 (socket says 172.21.15.3:59136) 2023-11-04T05:29:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f46987700 1 -- 172.21.15.3:0/362030252 learned_addr learned my addr 172.21.15.3:0/362030252 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f46186700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 0x7f9f400b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927495007 0 0) 0x7f9f400cb9d0 con 0x7f9f400ce130 2023-11-04T05:29:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c0035b0 con 0x7f9f400ce130 2023-11-04T05:29:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481173759 0 0) 0x7f9f2c0035b0 con 0x7f9f400ce130 2023-11-04T05:29:37.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 msgr2=0x7f9f400b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:37.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 0x7f9f400b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 >> v1:172.21.15.188:6789/0 conn(0x7f9f400cdb30 legacy=0x7f9f400cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400cd050 con 0x7f9f400ce130 2023-11-04T05:29:37.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f9f380019f0 con 0x7f9f400ce130 2023-11-04T05:29:37.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 >> v1:172.21.15.3:6789/0 conn(0x7f9f400ce130 legacy=0x7f9f400cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c003a30 0x7f9f2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c0043e0 0x7f9f2c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 0x7f9f2c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400ccd70 con 0x7f9f2c0043e0 2023-11-04T05:29:37.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c00ba10 con 0x7f9f2c003a30 2023-11-04T05:29:37.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c00bbd0 con 0x7f9f4006ad90 2023-11-04T05:29:37.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.591+0000 7f9f45985700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c0043e0 0x7f9f2c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f46186700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c003a30 0x7f9f2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f46987700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 0x7f9f2c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45985700 1 -- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 msgr2=0x7f9f2c005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:37.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45985700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f4006ad90 0x7f9f2c005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45985700 1 -- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c003a30 msgr2=0x7f9f2c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:37.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45985700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c003a30 0x7f9f2c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45985700 1 -- 172.21.15.3:0/362030252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0035b0 con 0x7f9f2c0043e0 2023-11-04T05:29:37.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45985700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c0043e0 0x7f9f2c0048b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9f400cdf90 tx=0x7f9f2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b339c2125f4bbd server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:37.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30016070 con 0x7f9f2c0043e0 2023-11-04T05:29:37.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30006860 con 0x7f9f2c0043e0 2023-11-04T05:29:37.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.592+0000 7f9f45184700 1 -- 172.21.15.3:0/362030252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30001d80 con 0x7f9f2c0043e0 2023-11-04T05:29:37.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f483ea700 1 -- 172.21.15.3:0/362030252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c0043e0 msgr2=0x7f9f2c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f483ea700 1 --2- 172.21.15.3:0/362030252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c0043e0 0x7f9f2c0048b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9f400cdf90 tx=0x7f9f2c005880 comp rx=0 tx=0).stop 2023-11-04T05:29:37.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f46987700 1 -- 172.21.15.3:0/362030252 reap_dead start 2023-11-04T05:29:37.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f483ea700 1 -- 172.21.15.3:0/362030252 shutdown_connections 2023-11-04T05:29:37.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f483ea700 1 -- 172.21.15.3:0/362030252 >> 172.21.15.3:0/362030252 conn(0x7f9f400d0560 msgr2=0x7f9f400d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:37.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f483ea700 1 -- 172.21.15.3:0/362030252 shutdown_connections 2023-11-04T05:29:37.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.593+0000 7f9f483ea700 1 -- 172.21.15.3:0/362030252 wait complete. 2023-11-04T05:29:37.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.594+0000 7f9f483ea700 1 Processor -- start 2023-11-04T05:29:37.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.594+0000 7f9f483ea700 1 -- start start 2023-11-04T05:29:37.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c0043e0 0x7f9f401f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400b00a0 0x7f9f400b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 0x7f9f400c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400b0a60 con 0x7f9f400c50e0 2023-11-04T05:29:37.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c5aa0 con 0x7f9f2c0043e0 2023-11-04T05:29:37.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c68d0 con 0x7f9f400b00a0 2023-11-04T05:29:37.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 0x7f9f400c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 0x7f9f400c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53800/0 (socket says 172.21.15.3:53800) 2023-11-04T05:29:37.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 -- 172.21.15.3:0/1445636911 learned_addr learned my addr 172.21.15.3:0/1445636911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:37.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f45985700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400b00a0 0x7f9f400b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46186700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c0043e0 0x7f9f401f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 -- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400b00a0 msgr2=0x7f9f400b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400b00a0 0x7f9f400b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 -- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c0043e0 msgr2=0x7f9f401f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c0043e0 0x7f9f401f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 -- 172.21.15.3:0/1445636911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f401f4560 con 0x7f9f400c50e0 2023-11-04T05:29:37.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f46987700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 0x7f9f400c5560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9f38001a40 tx=0x7f9f38007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:37.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38017070 con 0x7f9f400c50e0 2023-11-04T05:29:37.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3800ec20 con 0x7f9f400c50e0 2023-11-04T05:29:37.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800edd0 con 0x7f9f400c50e0 2023-11-04T05:29:37.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.595+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f401f3920 con 0x7f9f400c50e0 2023-11-04T05:29:37.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.596+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f400c5fa0 con 0x7f9f400c50e0 2023-11-04T05:29:37.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.596+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f400cc8c0 con 0x7f9f400c50e0 2023-11-04T05:29:37.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.597+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f38013420 con 0x7f9f400c50e0 2023-11-04T05:29:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.597+0000 7f9f36ffd700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9f2406af00 0x7f9f2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.597+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9f380890a0 con 0x7f9f400c50e0 2023-11-04T05:29:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.597+0000 7f9f46186700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9f2406af00 0x7f9f2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.598+0000 7f9f46186700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9f2406af00 0x7f9f2406d3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009b80 tx=0x7f9f3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.602+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f9f38054360 con 0x7f9f400c50e0 2023-11-04T05:29:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.887+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9f400c6a10 con 0x7f9f400c50e0 2023-11-04T05:29:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.891+0000 7f9f36ffd700 1 -- 172.21.15.3:0/1445636911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9f38057980 con 0x7f9f400c50e0 2023-11-04T05:29:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9f2406af00 msgr2=0x7f9f2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9f2406af00 0x7f9f2406d3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009b80 tx=0x7f9f3c008040 comp rx=0 tx=0).stop 2023-11-04T05:29:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 msgr2=0x7f9f400c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 0x7f9f400c5560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9f38001a40 tx=0x7f9f38007c90 comp rx=0 tx=0).stop 2023-11-04T05:29:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 shutdown_connections 2023-11-04T05:29:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f9f2406af00 0x7f9f2406d3c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c0043e0 0x7f9f401f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f400c50e0 0x7f9f400c5560 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 --2- 172.21.15.3:0/1445636911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400b00a0 0x7f9f400b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 >> 172.21.15.3:0/1445636911 conn(0x7f9f400d0560 msgr2=0x7f9f401e63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 shutdown_connections 2023-11-04T05:29:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:37.893+0000 7f9f483ea700 1 -- 172.21.15.3:0/1445636911 wait complete. 2023-11-04T05:29:37.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-04T05:29:38.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.031+0000 7f480a56b700 1 Processor -- start 2023-11-04T05:29:38.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.031+0000 7f480a56b700 1 -- start start 2023-11-04T05:29:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f480a56b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040c6c30 0x7f48040ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f480a56b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48040d1080 0x7f48040cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f480a56b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040c4830 con 0x7f48040c6850 2023-11-04T05:29:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f480a56b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040cc740 con 0x7f48040d1080 2023-11-04T05:29:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f480a56b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040cc460 con 0x7f48040c6c30 2023-11-04T05:29:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f4808b08700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48040d1080 0x7f48040cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f4803fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f48040c6850 0x7f48040cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59152/0 (socket says 172.21.15.3:59152) 2023-11-04T05:29:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.032+0000 7f4803fff700 1 -- 172.21.15.3:0/3310893611 learned_addr learned my addr 172.21.15.3:0/3310893611 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f48037fe700 1 --2- 172.21.15.3:0/3310893611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040c6c30 0x7f48040ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4808b08700 1 -- 172.21.15.3:0/3310893611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040c6c30 msgr2=0x7f48040ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4808b08700 1 --2- 172.21.15.3:0/3310893611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040c6c30 0x7f48040ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4808b08700 1 -- 172.21.15.3:0/3310893611 >> v1:172.21.15.3:6789/0 conn(0x7f48040c6850 legacy=0x7f48040cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4802ffd700 1 -- 172.21.15.3:0/3310893611 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229725371 0 0) 0x7f48040c4830 con 0x7f48040c6850 2023-11-04T05:29:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4808b08700 1 -- 172.21.15.3:0/3310893611 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040c41e0 con 0x7f48040d1080 2023-11-04T05:29:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4808b08700 1 --2- 172.21.15.3:0/3310893611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040d1080 0x7f48040cd300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009d90 tx=0x7f47fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b153a67b1314ad server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4802ffd700 1 -- 172.21.15.3:0/3310893611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00e070 con 0x7f48040d1080 2023-11-04T05:29:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4802ffd700 1 -- 172.21.15.3:0/3310893611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00bbd0 con 0x7f48040d1080 2023-11-04T05:29:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.033+0000 7f4802ffd700 1 -- 172.21.15.3:0/3310893611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc012430 con 0x7f48040d1080 2023-11-04T05:29:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 -- 172.21.15.3:0/3310893611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040d1080 msgr2=0x7f48040cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 --2- 172.21.15.3:0/3310893611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040d1080 0x7f48040cd300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009d90 tx=0x7f47fc00b330 comp rx=0 tx=0).stop 2023-11-04T05:29:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 -- 172.21.15.3:0/3310893611 shutdown_connections 2023-11-04T05:29:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 --2- 172.21.15.3:0/3310893611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040d1080 0x7f48040cd300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 --2- 172.21.15.3:0/3310893611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040c6c30 0x7f48040ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 -- 172.21.15.3:0/3310893611 >> 172.21.15.3:0/3310893611 conn(0x7f48040d0560 msgr2=0x7f48040d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 -- 172.21.15.3:0/3310893611 shutdown_connections 2023-11-04T05:29:38.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.034+0000 7f480a56b700 1 -- 172.21.15.3:0/3310893611 wait complete. 2023-11-04T05:29:38.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.035+0000 7f480a56b700 1 Processor -- start 2023-11-04T05:29:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.035+0000 7f480a56b700 1 -- start start 2023-11-04T05:29:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f480a56b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 0x7f480414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f480a56b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48040c6c30 0x7f48041f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f480a56b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040d1080 0x7f48041eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f480a56b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041ef300 con 0x7f48040c6850 2023-11-04T05:29:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f480a56b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041ef5a0 con 0x7f48040c6c30 2023-11-04T05:29:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f480a56b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48041ef880 con 0x7f48040d1080 2023-11-04T05:29:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 0x7f480414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 0x7f480414f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53810/0 (socket says 172.21.15.3:53810) 2023-11-04T05:29:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 -- 172.21.15.3:0/294374460 learned_addr learned my addr 172.21.15.3:0/294374460 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:38.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4808b08700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040d1080 0x7f48041eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f48037fe700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48040c6c30 0x7f48041f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 -- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040d1080 msgr2=0x7f48041eedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040d1080 0x7f48041eedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 -- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48040c6c30 msgr2=0x7f48041f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48040c6c30 0x7f48041f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 -- 172.21.15.3:0/294374460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc009a70 con 0x7f48040c6850 2023-11-04T05:29:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.036+0000 7f4803fff700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 0x7f480414f350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f47f40010e0 tx=0x7f47f400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.037+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f40074e0 con 0x7f48040c6850 2023-11-04T05:29:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.037+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4007640 con 0x7f48040c6850 2023-11-04T05:29:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.037+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48041efb60 con 0x7f48040c6850 2023-11-04T05:29:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.037+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4005a70 con 0x7f48040c6850 2023-11-04T05:29:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.037+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804156ca0 con 0x7f48040c6850 2023-11-04T05:29:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.038+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4804074df0 con 0x7f48040c6850 2023-11-04T05:29:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.038+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47f4005bd0 con 0x7f48040c6850 2023-11-04T05:29:38.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.039+0000 7f4800ff9700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f47f006af00 0x7f47f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.039+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f47f4088890 con 0x7f48040c6850 2023-11-04T05:29:38.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.039+0000 7f48037fe700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f47f006af00 0x7f47f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.039+0000 7f48037fe700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f47f006af00 0x7f47f006d3c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48040d03d0 tx=0x7f47f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.043+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f47f4053be0 con 0x7f48040c6850 2023-11-04T05:29:38.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.331+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f48041efd50 con 0x7f48040c6850 2023-11-04T05:29:38.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.335+0000 7f4800ff9700 1 -- 172.21.15.3:0/294374460 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f47f4017070 con 0x7f48040c6850 2023-11-04T05:29:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f47f006af00 msgr2=0x7f47f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f47f006af00 0x7f47f006d3c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48040d03d0 tx=0x7f47f8008040 comp rx=0 tx=0).stop 2023-11-04T05:29:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 msgr2=0x7f480414f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 0x7f480414f350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f47f40010e0 tx=0x7f47f400cae0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 shutdown_connections 2023-11-04T05:29:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f47f006af00 0x7f47f006d3c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48040d1080 0x7f48041eedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48040c6850 0x7f480414f350 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 --2- 172.21.15.3:0/294374460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48040c6c30 0x7f48041f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.337+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 >> 172.21.15.3:0/294374460 conn(0x7f48040d0560 msgr2=0x7f480406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.338+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 shutdown_connections 2023-11-04T05:29:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.338+0000 7f480a56b700 1 -- 172.21.15.3:0/294374460 wait complete. 2023-11-04T05:29:38.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-04T05:29:38.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.476+0000 7f5885b32700 1 Processor -- start 2023-11-04T05:29:38.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.476+0000 7f5885b32700 1 -- start start 2023-11-04T05:29:38.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.476+0000 7f5885b32700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 0x7f58800ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.476+0000 7f5885b32700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800c4830 con 0x7f58800c6850 2023-11-04T05:29:38.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.476+0000 7f5885b32700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800cc740 con 0x7f58800d1080 2023-11-04T05:29:38.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.476+0000 7f5885b32700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800cc460 con 0x7f58800c6c30 2023-11-04T05:29:38.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f58800c6850 0x7f58800cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59166/0 (socket says 172.21.15.3:59166) 2023-11-04T05:29:38.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587f7fe700 1 -- 172.21.15.3:0/1490354301 learned_addr learned my addr 172.21.15.3:0/1490354301 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:38.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587effd700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 0x7f58800ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702620013 0 0) 0x7f58800c4830 con 0x7f58800c6850 2023-11-04T05:29:38.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58680035b0 con 0x7f58800c6850 2023-11-04T05:29:38.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511844307 0 0) 0x7f58800cc740 con 0x7f58800d1080 2023-11-04T05:29:38.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58800c4830 con 0x7f58800d1080 2023-11-04T05:29:38.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4008268755 0 0) 0x7f58680035b0 con 0x7f58800c6850 2023-11-04T05:29:38.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 msgr2=0x7f58800ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:38.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 0x7f58800ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 >> v1:172.21.15.188:6789/0 conn(0x7f58800d1080 legacy=0x7f588015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800c41e0 con 0x7f58800c6850 2023-11-04T05:29:38.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f58700017f0 con 0x7f58800c6850 2023-11-04T05:29:38.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 >> v1:172.21.15.3:6789/0 conn(0x7f58800c6850 legacy=0x7f58800cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 0x7f5868003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5868004170 0x7f5868004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5868004d90 0x7f5868005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800cc460 con 0x7f5868004d90 2023-11-04T05:29:38.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586800bb50 con 0x7f5868004170 2023-11-04T05:29:38.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586800bd10 con 0x7f58800c6c30 2023-11-04T05:29:38.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587ffff700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5868004d90 0x7f5868005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587effd700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5868004170 0x7f5868004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.477+0000 7f587f7fe700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 0x7f5868003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 -- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 msgr2=0x7f5868003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:38.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800c6c30 0x7f5868003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 -- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5868004170 msgr2=0x7f5868004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:38.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5868004170 0x7f5868004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 -- 172.21.15.3:0/1490354301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58680035b0 con 0x7f5868004d90 2023-11-04T05:29:38.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5868004d90 0x7f5868005210 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f587800c2f0 tx=0x7f5868005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23d7eb39d9fd21e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878001f30 con 0x7f5868004d90 2023-11-04T05:29:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587800ce90 con 0x7f5868004d90 2023-11-04T05:29:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587e7fc700 1 -- 172.21.15.3:0/1490354301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878003c40 con 0x7f5868004d90 2023-11-04T05:29:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f5885b32700 1 -- 172.21.15.3:0/1490354301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5868004d90 msgr2=0x7f5868005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f5885b32700 1 --2- 172.21.15.3:0/1490354301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5868004d90 0x7f5868005210 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f587800c2f0 tx=0x7f5868005880 comp rx=0 tx=0).stop 2023-11-04T05:29:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.478+0000 7f587ffff700 1 -- 172.21.15.3:0/1490354301 reap_dead start 2023-11-04T05:29:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.479+0000 7f5885b32700 1 -- 172.21.15.3:0/1490354301 shutdown_connections 2023-11-04T05:29:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.479+0000 7f5885b32700 1 -- 172.21.15.3:0/1490354301 >> 172.21.15.3:0/1490354301 conn(0x7f58800d0560 msgr2=0x7f58800d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.479+0000 7f5885b32700 1 -- 172.21.15.3:0/1490354301 shutdown_connections 2023-11-04T05:29:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.479+0000 7f5885b32700 1 -- 172.21.15.3:0/1490354301 wait complete. 2023-11-04T05:29:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.480+0000 7f5885b32700 1 Processor -- start 2023-11-04T05:29:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.480+0000 7f5885b32700 1 -- start start 2023-11-04T05:29:38.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5885b32700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 0x7f58800ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5885b32700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58800cd0f0 0x7f58800aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5885b32700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800af450 0x7f58800b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5885b32700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800cd570 con 0x7f588015d710 2023-11-04T05:29:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5885b32700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800afeb0 con 0x7f58800cd0f0 2023-11-04T05:29:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5885b32700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58801f38e0 con 0x7f58800af450 2023-11-04T05:29:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 0x7f58800ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 0x7f58800ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53830/0 (socket says 172.21.15.3:53830) 2023-11-04T05:29:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 -- 172.21.15.3:0/2080888782 learned_addr learned my addr 172.21.15.3:0/2080888782 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 -- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800af450 msgr2=0x7f58800b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800af450 0x7f58800b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587effd700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58800cd0f0 0x7f58800aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 -- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58800cd0f0 msgr2=0x7f58800aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58800cd0f0 0x7f58800aef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 -- 172.21.15.3:0/2080888782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800afbd0 con 0x7f588015d710 2023-11-04T05:29:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f587f7fe700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 0x7f58800ccbb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f587000f910 tx=0x7f587000fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.481+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700072e0 con 0x7f588015d710 2023-11-04T05:29:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.482+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5870007440 con 0x7f588015d710 2023-11-04T05:29:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.482+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58801f3290 con 0x7f588015d710 2023-11-04T05:29:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.482+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870005830 con 0x7f588015d710 2023-11-04T05:29:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.482+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801f2fa0 con 0x7f588015d710 2023-11-04T05:29:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.482+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588006abc0 con 0x7f588015d710 2023-11-04T05:29:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.483+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5870005990 con 0x7f588015d710 2023-11-04T05:29:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.483+0000 7f5867fff700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f586c06af00 0x7f586c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.484+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5870089410 con 0x7f588015d710 2023-11-04T05:29:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.484+0000 7f587effd700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f586c06af00 0x7f586c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.484+0000 7f587effd700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f586c06af00 0x7f586c06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58800b2e30 tx=0x7f5874008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.488+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f58700546d0 con 0x7f588015d710 2023-11-04T05:29:38.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.772+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5880074df0 con 0x7f588015d710 2023-11-04T05:29:38.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.774+0000 7f5867fff700 1 -- 172.21.15.3:0/2080888782 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5870057cf0 con 0x7f588015d710 2023-11-04T05:29:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f586c06af00 msgr2=0x7f586c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f586c06af00 0x7f586c06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58800b2e30 tx=0x7f5874008040 comp rx=0 tx=0).stop 2023-11-04T05:29:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 msgr2=0x7f58800ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 0x7f58800ccbb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f587000f910 tx=0x7f587000fcd0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 shutdown_connections 2023-11-04T05:29:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f586c06af00 0x7f586c06d3c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58800af450 0x7f58800b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f588015d710 0x7f58800ccbb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 --2- 172.21.15.3:0/2080888782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58800cd0f0 0x7f58800aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.776+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 >> 172.21.15.3:0/2080888782 conn(0x7f58800d0560 msgr2=0x7f58800cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.777+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 shutdown_connections 2023-11-04T05:29:38.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.777+0000 7f5885b32700 1 -- 172.21.15.3:0/2080888782 wait complete. 2023-11-04T05:29:38.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-04T05:29:38.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.915+0000 7fa0fb789700 1 Processor -- start 2023-11-04T05:29:38.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.915+0000 7fa0fb789700 1 -- start start 2023-11-04T05:29:38.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0fb789700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0f40cdb30 0x7fa0f415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0fb789700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f40b1ce0 con 0x7fa0f406ad90 2023-11-04T05:29:38.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0fb789700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f40ccd70 con 0x7fa0f40ce130 2023-11-04T05:29:38.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0fb789700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f40cd050 con 0x7fa0f40cdb30 2023-11-04T05:29:38.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0f8d24700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0f40cdb30 0x7fa0f415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0f9525700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa0f406ad90 0x7fa0f40b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59186/0 (socket says 172.21.15.3:59186) 2023-11-04T05:29:38.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0f9525700 1 -- 172.21.15.3:0/3305852122 learned_addr learned my addr 172.21.15.3:0/3305852122 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:38.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.916+0000 7fa0ebfff700 1 -- 172.21.15.3:0/3305852122 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750290250 0 0) 0x7fa0f40b1ce0 con 0x7fa0f406ad90 2023-11-04T05:29:38.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0ebfff700 1 -- 172.21.15.3:0/3305852122 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e00035b0 con 0x7fa0f406ad90 2023-11-04T05:29:38.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0f8d24700 1 -- 172.21.15.3:0/3305852122 >> v1:172.21.15.188:6789/0 conn(0x7fa0f40ce130 legacy=0x7fa0f415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:38.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0f8d24700 1 -- 172.21.15.3:0/3305852122 >> v1:172.21.15.3:6789/0 conn(0x7fa0f406ad90 legacy=0x7fa0f40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0f8d24700 1 -- 172.21.15.3:0/3305852122 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f40cd330 con 0x7fa0f40cdb30 2023-11-04T05:29:38.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0f8d24700 1 --2- 172.21.15.3:0/3305852122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415d1e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009840 tx=0x7fa0f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6f2b405145e688 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0ebfff700 1 -- 172.21.15.3:0/3305852122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0004230 con 0x7fa0f40cdb30 2023-11-04T05:29:38.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0ebfff700 1 -- 172.21.15.3:0/3305852122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0004390 con 0x7fa0f40cdb30 2023-11-04T05:29:38.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0ebfff700 1 -- 172.21.15.3:0/3305852122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00036c0 con 0x7fa0f40cdb30 2023-11-04T05:29:38.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0fb789700 1 -- 172.21.15.3:0/3305852122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 msgr2=0x7fa0f415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0fb789700 1 --2- 172.21.15.3:0/3305852122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415d1e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009840 tx=0x7fa0f0009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:38.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0fb789700 1 -- 172.21.15.3:0/3305852122 shutdown_connections 2023-11-04T05:29:38.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0fb789700 1 --2- 172.21.15.3:0/3305852122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415d1e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.917+0000 7fa0fb789700 1 -- 172.21.15.3:0/3305852122 >> 172.21.15.3:0/3305852122 conn(0x7fa0f40d0560 msgr2=0x7fa0f40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:38.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.918+0000 7fa0fb789700 1 -- 172.21.15.3:0/3305852122 shutdown_connections 2023-11-04T05:29:38.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.918+0000 7fa0fb789700 1 -- 172.21.15.3:0/3305852122 wait complete. 2023-11-04T05:29:38.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.919+0000 7fa0fb789700 1 Processor -- start 2023-11-04T05:29:38.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.919+0000 7fa0fb789700 1 -- start start 2023-11-04T05:29:38.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0fb789700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f406ad90 0x7fa0f40b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0fb789700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0fb789700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f40ce130 0x7fa0f4158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0fb789700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40b49f0 con 0x7fa0f40cdb30 2023-11-04T05:29:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0fb789700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f415f160 con 0x7fa0f40ce130 2023-11-04T05:29:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0fb789700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f415e8c0 con 0x7fa0f406ad90 2023-11-04T05:29:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0f8d24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0f8d24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53846/0 (socket says 172.21.15.3:53846) 2023-11-04T05:29:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0f8d24700 1 -- 172.21.15.3:0/2613253687 learned_addr learned my addr 172.21.15.3:0/2613253687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0f9525700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f406ad90 0x7fa0f40b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.920+0000 7fa0f9d26700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f40ce130 0x7fa0f4158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0f8d24700 1 -- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f406ad90 msgr2=0x7fa0f40b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0f8d24700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f406ad90 0x7fa0f40b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0f8d24700 1 -- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f40ce130 msgr2=0x7fa0f4158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0f8d24700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f40ce130 0x7fa0f4158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0f8d24700 1 -- 172.21.15.3:0/2613253687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00094a0 con 0x7fa0f40cdb30 2023-11-04T05:29:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0f8d24700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415e1c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00090b0 tx=0x7fa0f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011630 con 0x7fa0f40cdb30 2023-11-04T05:29:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0011790 con 0x7fa0f40cdb30 2023-11-04T05:29:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f415eba0 con 0x7fa0f40cdb30 2023-11-04T05:29:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011b00 con 0x7fa0f40cdb30 2023-11-04T05:29:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.921+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f415f8d0 con 0x7fa0f40cdb30 2023-11-04T05:29:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.922+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f40ad7f0 con 0x7fa0f40cdb30 2023-11-04T05:29:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.922+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0f0018040 con 0x7fa0f40cdb30 2023-11-04T05:29:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.923+0000 7fa0e9ffb700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa0d806af00 0x7fa0d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.923+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa0f0089c70 con 0x7fa0f40cdb30 2023-11-04T05:29:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.923+0000 7fa0f9525700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa0d806af00 0x7fa0d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:38.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.924+0000 7fa0f9525700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa0d806af00 0x7fa0d806d3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4001360 tx=0x7fa0e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:38.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:38.927+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fa0f0054fc0 con 0x7fa0f40cdb30 2023-11-04T05:29:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.212+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa0f40b0da0 con 0x7fa0f40cdb30 2023-11-04T05:29:39.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.214+0000 7fa0e9ffb700 1 -- 172.21.15.3:0/2613253687 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa0f002b0a0 con 0x7fa0f40cdb30 2023-11-04T05:29:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa0d806af00 msgr2=0x7fa0d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa0d806af00 0x7fa0d806d3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4001360 tx=0x7fa0e4009040 comp rx=0 tx=0).stop 2023-11-04T05:29:39.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 msgr2=0x7fa0f415e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415e1c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00090b0 tx=0x7fa0f0004750 comp rx=0 tx=0).stop 2023-11-04T05:29:39.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 shutdown_connections 2023-11-04T05:29:39.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fa0d806af00 0x7fa0d806d3c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f40ce130 0x7fa0f4158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f406ad90 0x7fa0f40b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 --2- 172.21.15.3:0/2613253687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0f40cdb30 0x7fa0f415e1c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 >> 172.21.15.3:0/2613253687 conn(0x7fa0f40d0560 msgr2=0x7fa0f40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:39.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 shutdown_connections 2023-11-04T05:29:39.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.217+0000 7fa0fb789700 1 -- 172.21.15.3:0/2613253687 wait complete. 2023-11-04T05:29:39.231 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:39.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.354+0000 7f7025604700 1 Processor -- start 2023-11-04T05:29:39.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.354+0000 7f7025604700 1 -- start start 2023-11-04T05:29:39.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f7025604700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 0x7f702006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f7025604700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70200d2690 0x7f70200d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f7025604700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70201616a0 con 0x7f70200b2e00 2023-11-04T05:29:39.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f7025604700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70200b2bb0 con 0x7f70200d2690 2023-11-04T05:29:39.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f7025604700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70200d42c0 con 0x7f70201612c0 2023-11-04T05:29:39.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f70200b2e00 0x7f70200d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59200/0 (socket says 172.21.15.3:59200) 2023-11-04T05:29:39.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701e7fc700 1 -- 172.21.15.3:0/3535983948 learned_addr learned my addr 172.21.15.3:0/3535983948 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:39.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701effd700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 0x7f702006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701f7fe700 1 --2- 172.21.15.3:0/3535983948 >> v2:172.21.15.188:3300/0 conn(0x7f70200d2690 0x7f70200d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294090548 0 0) 0x7f70201616a0 con 0x7f70200b2e00 2023-11-04T05:29:39.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70080035b0 con 0x7f70200b2e00 2023-11-04T05:29:39.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030857925 0 0) 0x7f70080035b0 con 0x7f70200b2e00 2023-11-04T05:29:39.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 msgr2=0x7f702006bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:39.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 0x7f702006bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 >> v2:172.21.15.188:3300/0 conn(0x7f70200d2690 msgr2=0x7f70200d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:39.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 --2- 172.21.15.3:0/3535983948 >> v2:172.21.15.188:3300/0 conn(0x7f70200d2690 0x7f70200d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200d45a0 con 0x7f70200b2e00 2023-11-04T05:29:39.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f7010001670 con 0x7f70200b2e00 2023-11-04T05:29:39.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.355+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 >> v1:172.21.15.3:6789/0 conn(0x7f70200b2e00 legacy=0x7f70200d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 0x7f7008003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7008003f80 0x7f7008004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7008004d90 0x7f7008005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200b2bb0 con 0x7f7008004d90 2023-11-04T05:29:39.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200d42c0 con 0x7f7008003f80 2023-11-04T05:29:39.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700800ba60 con 0x7f70201612c0 2023-11-04T05:29:39.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7008004d90 0x7f7008005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701effd700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 0x7f7008003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701e7fc700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7008003f80 0x7f7008004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 -- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 msgr2=0x7f7008003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70201612c0 0x7f7008003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 -- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7008003f80 msgr2=0x7f7008004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7008003f80 0x7f7008004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 -- 172.21.15.3:0/3535983948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080035b0 con 0x7f7008004d90 2023-11-04T05:29:39.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701f7fe700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7008004d90 0x7f7008005210 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f701800e170 tx=0x7f7008005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bfafa70c4bf3a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:39.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800ec90 con 0x7f7008004d90 2023-11-04T05:29:39.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701800f040 con 0x7f7008004d90 2023-11-04T05:29:39.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.356+0000 7f701dffb700 1 -- 172.21.15.3:0/3535983948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018014480 con 0x7f7008004d90 2023-11-04T05:29:39.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f7025604700 1 -- 172.21.15.3:0/3535983948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7008004d90 msgr2=0x7f7008005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f7025604700 1 --2- 172.21.15.3:0/3535983948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7008004d90 0x7f7008005210 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f701800e170 tx=0x7f7008005880 comp rx=0 tx=0).stop 2023-11-04T05:29:39.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f701f7fe700 1 -- 172.21.15.3:0/3535983948 reap_dead start 2023-11-04T05:29:39.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f7025604700 1 -- 172.21.15.3:0/3535983948 shutdown_connections 2023-11-04T05:29:39.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f7025604700 1 -- 172.21.15.3:0/3535983948 >> 172.21.15.3:0/3535983948 conn(0x7f702008aff0 msgr2=0x7f70200d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f7025604700 1 -- 172.21.15.3:0/3535983948 shutdown_connections 2023-11-04T05:29:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.357+0000 7f7025604700 1 -- 172.21.15.3:0/3535983948 wait complete. 2023-11-04T05:29:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.358+0000 7f7025604700 1 Processor -- start 2023-11-04T05:29:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.358+0000 7f7025604700 1 -- start start 2023-11-04T05:29:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f7025604700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 0x7f702015ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f7025604700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f702015ef70 0x7f702006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f7025604700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f702006cd90 0x7f70200b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f7025604700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702006d220 con 0x7f70200cd950 2023-11-04T05:29:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f7025604700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702015d4e0 con 0x7f702006cd90 2023-11-04T05:29:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f7025604700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702015d7c0 con 0x7f702015ef70 2023-11-04T05:29:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 0x7f702015ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 0x7f702015ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53864/0 (socket says 172.21.15.3:53864) 2023-11-04T05:29:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 -- 172.21.15.3:0/2181345951 learned_addr learned my addr 172.21.15.3:0/2181345951 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701f7fe700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f702006cd90 0x7f70200b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701e7fc700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f702015ef70 0x7f702006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 -- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f702015ef70 msgr2=0x7f702006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f702015ef70 0x7f702006c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 -- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f702006cd90 msgr2=0x7f70200b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f702006cd90 0x7f70200b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701effd700 1 -- 172.21.15.3:0/2181345951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702015daa0 con 0x7f70200cd950 2023-11-04T05:29:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701f7fe700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f702006cd90 0x7f70200b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.359+0000 7f701e7fc700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f702015ef70 0x7f702006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.360+0000 7f701effd700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 0x7f702015ce50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f70140119f0 tx=0x7f7014011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.360+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014005c30 con 0x7f70200cd950 2023-11-04T05:29:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.360+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014005d90 con 0x7f70200cd950 2023-11-04T05:29:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.360+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702015dd80 con 0x7f70200cd950 2023-11-04T05:29:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.360+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400f950 con 0x7f70200cd950 2023-11-04T05:29:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.360+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70200cdd30 con 0x7f70200cd950 2023-11-04T05:29:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.361+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7020074df0 con 0x7f70200cd950 2023-11-04T05:29:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.361+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f701400fab0 con 0x7f70200cd950 2023-11-04T05:29:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.362+0000 7f70077fe700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f700c06af00 0x7f700c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.362+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7014089bd0 con 0x7f70200cd950 2023-11-04T05:29:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.362+0000 7f701e7fc700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f700c06af00 0x7f700c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.363+0000 7f701e7fc700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f700c06af00 0x7f700c06d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70100048d0 tx=0x7f7010009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.366+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7014054e90 con 0x7f70200cd950 2023-11-04T05:29:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.569+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70200d1d60 con 0x7f70200cd950 2023-11-04T05:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.570+0000 7f70077fe700 1 -- 172.21.15.3:0/2181345951 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13002 (secure 0 0 0) 0x7f7014021020 con 0x7f70200cd950 2023-11-04T05:29:39.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:39.573 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:36.096707+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f700c06af00 msgr2=0x7f700c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f700c06af00 0x7f700c06d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70100048d0 tx=0x7f7010009040 comp rx=0 tx=0).stop 2023-11-04T05:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 msgr2=0x7f702015ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 0x7f702015ce50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f70140119f0 tx=0x7f7014011db0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 shutdown_connections 2023-11-04T05:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f700c06af00 0x7f700c06d3c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f702006cd90 0x7f70200b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70200cd950 0x7f702015ce50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 --2- 172.21.15.3:0/2181345951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f702015ef70 0x7f702006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 >> 172.21.15.3:0/2181345951 conn(0x7f702008aff0 msgr2=0x7f70200b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 shutdown_connections 2023-11-04T05:29:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.573+0000 7f7025604700 1 -- 172.21.15.3:0/2181345951 wait complete. 2023-11-04T05:29:39.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-04T05:29:39.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T05:29:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.714+0000 7f7585bb7700 1 Processor -- start 2023-11-04T05:29:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.714+0000 7f7585bb7700 1 -- start start 2023-11-04T05:29:39.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f7585bb7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75800ceaa0 0x7f75800cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f7585bb7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f758015b430 0x7f75800caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f7585bb7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758015b870 con 0x7f75800cee80 2023-11-04T05:29:39.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f7585bb7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75800ccc70 con 0x7f758015b430 2023-11-04T05:29:39.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f7585bb7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75800cdee0 con 0x7f75800ceaa0 2023-11-04T05:29:39.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f758015b430 0x7f75800caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f758015b430 0x7f75800caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53876/0 (socket says 172.21.15.3:53876) 2023-11-04T05:29:39.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 -- 172.21.15.3:0/8640668 learned_addr learned my addr 172.21.15.3:0/8640668 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:39.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757f7fe700 1 --2- 172.21.15.3:0/8640668 >> v2:172.21.15.188:3300/0 conn(0x7f75800ceaa0 0x7f75800cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 -- 172.21.15.3:0/8640668 >> v2:172.21.15.188:3300/0 conn(0x7f75800ceaa0 msgr2=0x7f75800cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 --2- 172.21.15.3:0/8640668 >> v2:172.21.15.188:3300/0 conn(0x7f75800ceaa0 0x7f75800cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 -- 172.21.15.3:0/8640668 >> v1:172.21.15.188:6789/0 conn(0x7f75800cee80 legacy=0x7f758015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 -- 172.21.15.3:0/8640668 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758015baa0 con 0x7f758015b430 2023-11-04T05:29:39.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757ffff700 1 --2- 172.21.15.3:0/8640668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f758015b430 0x7f75800caff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7580072c40 tx=0x7f757800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d940b7b2eac98ba server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.715+0000 7f757e7fc700 1 -- 172.21.15.3:0/8640668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578010070 con 0x7f758015b430 2023-11-04T05:29:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f757e7fc700 1 -- 172.21.15.3:0/8640668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757800dd50 con 0x7f758015b430 2023-11-04T05:29:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f757e7fc700 1 -- 172.21.15.3:0/8640668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757800b310 con 0x7f758015b430 2023-11-04T05:29:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 -- 172.21.15.3:0/8640668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f758015b430 msgr2=0x7f75800caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 --2- 172.21.15.3:0/8640668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f758015b430 0x7f75800caff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7580072c40 tx=0x7f757800d430 comp rx=0 tx=0).stop 2023-11-04T05:29:39.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 -- 172.21.15.3:0/8640668 shutdown_connections 2023-11-04T05:29:39.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 --2- 172.21.15.3:0/8640668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f758015b430 0x7f75800caff0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 --2- 172.21.15.3:0/8640668 >> v2:172.21.15.188:3300/0 conn(0x7f75800ceaa0 0x7f75800cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 -- 172.21.15.3:0/8640668 >> 172.21.15.3:0/8640668 conn(0x7f7580072560 msgr2=0x7f7580099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:39.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 -- 172.21.15.3:0/8640668 shutdown_connections 2023-11-04T05:29:39.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.716+0000 7f7585bb7700 1 -- 172.21.15.3:0/8640668 wait complete. 2023-11-04T05:29:39.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.717+0000 7f7585bb7700 1 Processor -- start 2023-11-04T05:29:39.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 -- start start 2023-11-04T05:29:39.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 0x7f7580154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75800cee80 0x7f75800c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758015b430 0x7f758014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800ca580 con 0x7f75800ceaa0 2023-11-04T05:29:39.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800c93d0 con 0x7f75800cee80 2023-11-04T05:29:39.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f7585bb7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758014afd0 con 0x7f758015b430 2023-11-04T05:29:39.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 0x7f7580154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 0x7f7580154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53878/0 (socket says 172.21.15.3:53878) 2023-11-04T05:29:39.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.718+0000 7f757f7fe700 1 -- 172.21.15.3:0/1619677833 learned_addr learned my addr 172.21.15.3:0/1619677833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:39.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757effd700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75800cee80 0x7f75800c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757ffff700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758015b430 0x7f758014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757f7fe700 1 -- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758015b430 msgr2=0x7f758014aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757f7fe700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758015b430 0x7f758014aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757f7fe700 1 -- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75800cee80 msgr2=0x7f75800c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757f7fe700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75800cee80 0x7f75800c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757f7fe700 1 -- 172.21.15.3:0/1619677833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757800bd40 con 0x7f75800ceaa0 2023-11-04T05:29:39.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f757f7fe700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 0x7f7580154ad0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f757000d800 tx=0x7f757000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:39.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700125f0 con 0x7f75800ceaa0 2023-11-04T05:29:39.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570012750 con 0x7f75800ceaa0 2023-11-04T05:29:39.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75800c9fc0 con 0x7f75800ceaa0 2023-11-04T05:29:39.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570012b00 con 0x7f75800ceaa0 2023-11-04T05:29:39.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.719+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75800c8970 con 0x7f75800ceaa0 2023-11-04T05:29:39.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.720+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75800c8b30 con 0x7f75800ceaa0 2023-11-04T05:29:39.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.721+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7570012c60 con 0x7f75800ceaa0 2023-11-04T05:29:39.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.721+0000 7f7567fff700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f756c06af00 0x7f756c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:39.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.721+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7570089370 con 0x7f75800ceaa0 2023-11-04T05:29:39.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.721+0000 7f757effd700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f756c06af00 0x7f756c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:39.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.722+0000 7f757effd700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f756c06af00 0x7f756c06d3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f758006fff0 tx=0x7f7574007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:39.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.726+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f7570054770 con 0x7f75800ceaa0 2023-11-04T05:29:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.926+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7580074df0 con 0x7f75800ceaa0 2023-11-04T05:29:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.926+0000 7f7567fff700 1 -- 172.21.15.3:0/1619677833 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7570057d90 con 0x7f75800ceaa0 2023-11-04T05:29:39.930 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T05:29:39.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f756c06af00 msgr2=0x7f756c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f756c06af00 0x7f756c06d3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f758006fff0 tx=0x7f7574007ec0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 msgr2=0x7f7580154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 0x7f7580154ad0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f757000d800 tx=0x7f757000db10 comp rx=0 tx=0).stop 2023-11-04T05:29:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 shutdown_connections 2023-11-04T05:29:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f756c06af00 0x7f756c06d3c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758015b430 0x7f758014aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75800ceaa0 0x7f7580154ad0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 --2- 172.21.15.3:0/1619677833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75800cee80 0x7f75800c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 >> 172.21.15.3:0/1619677833 conn(0x7f7580072560 msgr2=0x7f758015a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 shutdown_connections 2023-11-04T05:29:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:39.929+0000 7f7585bb7700 1 -- 172.21.15.3:0/1619677833 wait complete. 2023-11-04T05:29:39.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-04T05:29:40.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.066+0000 7f1c76c39700 1 Processor -- start 2023-11-04T05:29:40.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.066+0000 7f1c76c39700 1 -- start start 2023-11-04T05:29:40.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c76c39700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c70159310 0x7f1c700cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c76c39700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c700716c0 0x7f1c7015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c76c39700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7015c280 con 0x7f1c7006d790 2023-11-04T05:29:40.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c76c39700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7015bf80 con 0x7f1c700716c0 2023-11-04T05:29:40.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c76c39700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c70071eb0 con 0x7f1c70159310 2023-11-04T05:29:40.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c6ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c700716c0 0x7f1c7015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c751d6700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1c7006d790 0x7f1c7015f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59238/0 (socket says 172.21.15.3:59238) 2023-11-04T05:29:40.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c751d6700 1 -- 172.21.15.3:0/2048713526 learned_addr learned my addr 172.21.15.3:0/2048713526 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:40.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c749d5700 1 --2- 172.21.15.3:0/2048713526 >> v2:172.21.15.188:3300/0 conn(0x7f1c70159310 0x7f1c700cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c6ffff700 1 -- 172.21.15.3:0/2048713526 >> v2:172.21.15.188:3300/0 conn(0x7f1c70159310 msgr2=0x7f1c700cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:40.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c6ffff700 1 --2- 172.21.15.3:0/2048713526 >> v2:172.21.15.188:3300/0 conn(0x7f1c70159310 0x7f1c700cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.067+0000 7f1c6ffff700 1 -- 172.21.15.3:0/2048713526 >> v1:172.21.15.3:6789/0 conn(0x7f1c7006d790 legacy=0x7f1c7015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c6ffff700 1 -- 172.21.15.3:0/2048713526 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c70072190 con 0x7f1c700716c0 2023-11-04T05:29:40.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c6ffff700 1 --2- 172.21.15.3:0/2048713526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c700716c0 0x7f1c7015c710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1c700d0ae0 tx=0x7f1c5c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2f9343e81f5e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:40.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c6f7fe700 1 -- 172.21.15.3:0/2048713526 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c005e30 con 0x7f1c700716c0 2023-11-04T05:29:40.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c6f7fe700 1 -- 172.21.15.3:0/2048713526 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c004030 con 0x7f1c700716c0 2023-11-04T05:29:40.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c6f7fe700 1 -- 172.21.15.3:0/2048713526 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0043c0 con 0x7f1c700716c0 2023-11-04T05:29:40.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 -- 172.21.15.3:0/2048713526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c700716c0 msgr2=0x7f1c7015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 --2- 172.21.15.3:0/2048713526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c700716c0 0x7f1c7015c710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1c700d0ae0 tx=0x7f1c5c009b00 comp rx=0 tx=0).stop 2023-11-04T05:29:40.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 -- 172.21.15.3:0/2048713526 shutdown_connections 2023-11-04T05:29:40.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 --2- 172.21.15.3:0/2048713526 >> v2:172.21.15.188:3300/0 conn(0x7f1c70159310 0x7f1c700cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 --2- 172.21.15.3:0/2048713526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c700716c0 0x7f1c7015c710 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 -- 172.21.15.3:0/2048713526 >> 172.21.15.3:0/2048713526 conn(0x7f1c700cfd50 msgr2=0x7f1c700d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:40.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 -- 172.21.15.3:0/2048713526 shutdown_connections 2023-11-04T05:29:40.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.068+0000 7f1c76c39700 1 -- 172.21.15.3:0/2048713526 wait complete. 2023-11-04T05:29:40.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.070+0000 7f1c76c39700 1 Processor -- start 2023-11-04T05:29:40.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.070+0000 7f1c76c39700 1 -- start start 2023-11-04T05:29:40.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c7006d790 0x7f1c701f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c700716c0 0x7f1c701e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 0x7f1c701e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c700cf8a0 con 0x7f1c70159310 2023-11-04T05:29:40.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c700c7b80 con 0x7f1c700716c0 2023-11-04T05:29:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c700b3540 con 0x7f1c7006d790 2023-11-04T05:29:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 0x7f1c701e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 0x7f1c701e5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53896/0 (socket says 172.21.15.3:53896) 2023-11-04T05:29:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 -- 172.21.15.3:0/735511781 learned_addr learned my addr 172.21.15.3:0/735511781 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:40.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c749d5700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c7006d790 0x7f1c701f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c6ffff700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c700716c0 0x7f1c701e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 -- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c7006d790 msgr2=0x7f1c701f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c7006d790 0x7f1c701f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 -- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c700716c0 msgr2=0x7f1c701e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c700716c0 0x7f1c701e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 -- 172.21.15.3:0/735511781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009430 con 0x7f1c70159310 2023-11-04T05:29:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c751d6700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 0x7f1c701e5d70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1c68002000 tx=0x7f1c6800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800ad40 con 0x7f1c70159310 2023-11-04T05:29:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680070d0 con 0x7f1c70159310 2023-11-04T05:29:40.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c700b3820 con 0x7f1c70159310 2023-11-04T05:29:40.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800aea0 con 0x7f1c70159310 2023-11-04T05:29:40.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.071+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c700c7e20 con 0x7f1c70159310 2023-11-04T05:29:40.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.072+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c700b3d40 con 0x7f1c70159310 2023-11-04T05:29:40.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.073+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c68007480 con 0x7f1c70159310 2023-11-04T05:29:40.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.073+0000 7f1c6d7fa700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c5406af00 0x7f1c5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.073+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1c68089320 con 0x7f1c70159310 2023-11-04T05:29:40.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.073+0000 7f1c749d5700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c5406af00 0x7f1c5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.074+0000 7f1c749d5700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c5406af00 0x7f1c5406d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400b9b0 tx=0x7f1c64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:40.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.078+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1c700b3d40 con 0x7f1c70159310 2023-11-04T05:29:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.284+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1c700b3d40 con 0x7f1c70159310 2023-11-04T05:29:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.284+0000 7f1c6d7fa700 1 -- 172.21.15.3:0/735511781 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1c68018060 con 0x7f1c70159310 2023-11-04T05:29:40.287 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:40.287 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-11-04T05:29:40.287 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-11-04T05:29:40.287 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-11-04T05:29:40.288 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-11-04T05:29:40.288 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-11-04T05:29:40.288 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-11-04T05:29:40.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c5406af00 msgr2=0x7f1c5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c5406af00 0x7f1c5406d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400b9b0 tx=0x7f1c64008040 comp rx=0 tx=0).stop 2023-11-04T05:29:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 msgr2=0x7f1c701e5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 0x7f1c701e5d70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1c68002000 tx=0x7f1c6800a340 comp rx=0 tx=0).stop 2023-11-04T05:29:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 shutdown_connections 2023-11-04T05:29:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1c5406af00 0x7f1c5406d3c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c70159310 0x7f1c701e5d70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c7006d790 0x7f1c701f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 --2- 172.21.15.3:0/735511781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c700716c0 0x7f1c701e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 >> 172.21.15.3:0/735511781 conn(0x7f1c700cfd50 msgr2=0x7f1c700b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 shutdown_connections 2023-11-04T05:29:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.286+0000 7f1c76c39700 1 -- 172.21.15.3:0/735511781 wait complete. 2023-11-04T05:29:40.300 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-04T05:29:40.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-04T05:29:40.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.428+0000 7f357d81b700 1 Processor -- start 2023-11-04T05:29:40.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.428+0000 7f357d81b700 1 -- start start 2023-11-04T05:29:40.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.428+0000 7f357d81b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35780d2800 0x7f35780d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.428+0000 7f357d81b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35780d1510 con 0x7f357815db50 2023-11-04T05:29:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.428+0000 7f357d81b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35780d1760 con 0x7f35780cf790 2023-11-04T05:29:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.428+0000 7f357d81b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35780d1a40 con 0x7f35780d2800 2023-11-04T05:29:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3576ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f357815db50 0x7f35780cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59248/0 (socket says 172.21.15.3:59248) 2023-11-04T05:29:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3576ffd700 1 -- 172.21.15.3:0/2295600835 learned_addr learned my addr 172.21.15.3:0/2295600835 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556684569 0 0) 0x7f35780d1510 con 0x7f357815db50 2023-11-04T05:29:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35600035b0 con 0x7f357815db50 2023-11-04T05:29:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f35767fc700 1 --2- 172.21.15.3:0/2295600835 >> v2:172.21.15.188:3300/0 conn(0x7f35780d2800 0x7f35780d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260053763 0 0) 0x7f35600035b0 con 0x7f357815db50 2023-11-04T05:29:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 >> v2:172.21.15.188:3300/0 conn(0x7f35780d2800 msgr2=0x7f35780d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 --2- 172.21.15.3:0/2295600835 >> v2:172.21.15.188:3300/0 conn(0x7f35780d2800 0x7f35780d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 >> v1:172.21.15.188:6789/0 conn(0x7f35780cf790 legacy=0x7f3578167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780d1d20 con 0x7f357815db50 2023-11-04T05:29:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f356c001670 con 0x7f357815db50 2023-11-04T05:29:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 >> v1:172.21.15.3:6789/0 conn(0x7f357815db50 legacy=0x7f35780cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3560003a00 0x7f3560003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3560004850 0x7f3560004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f356000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780d1a40 con 0x7f3560005210 2023-11-04T05:29:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356000c0b0 con 0x7f3560004850 2023-11-04T05:29:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356000c270 con 0x7f3560003a00 2023-11-04T05:29:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.429+0000 7f35777fe700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f356000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f3576ffd700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3560003a00 0x7f3560003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35767fc700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3560004850 0x7f3560004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 -- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3560003a00 msgr2=0x7f3560003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:40.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3560003a00 0x7f3560003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 -- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3560004850 msgr2=0x7f3560004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3560004850 0x7f3560004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 -- 172.21.15.3:0/2295600835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600035b0 con 0x7f3560005210 2023-11-04T05:29:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 -- 172.21.15.3:0/2295600835 reap_dead start 2023-11-04T05:29:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f35777fe700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f356000ba10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f357000c190 tx=0x7f3560005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af93a240ccb6eb14 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570001db0 con 0x7f3560005210 2023-11-04T05:29:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357000ccf0 con 0x7f3560005210 2023-11-04T05:29:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.430+0000 7f3575ffb700 1 -- 172.21.15.3:0/2295600835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570003c40 con 0x7f3560005210 2023-11-04T05:29:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 -- 172.21.15.3:0/2295600835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 msgr2=0x7f356000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f356000ba10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f357000c190 tx=0x7f3560005880 comp rx=0 tx=0).stop 2023-11-04T05:29:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 -- 172.21.15.3:0/2295600835 shutdown_connections 2023-11-04T05:29:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 --2- 172.21.15.3:0/2295600835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f356000ba10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 -- 172.21.15.3:0/2295600835 >> 172.21.15.3:0/2295600835 conn(0x7f357808aff0 msgr2=0x7f35780d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 -- 172.21.15.3:0/2295600835 shutdown_connections 2023-11-04T05:29:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.431+0000 7f357d81b700 1 -- 172.21.15.3:0/2295600835 wait complete. 2023-11-04T05:29:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.432+0000 7f357d81b700 1 Processor -- start 2023-11-04T05:29:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.432+0000 7f357d81b700 1 -- start start 2023-11-04T05:29:40.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f357d81b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f35781f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f357d81b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f357815ee40 0x7f35780c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f357d81b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35780b4380 0x7f35780b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f357d81b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35781528a0 con 0x7f3560005210 2023-11-04T05:29:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f357d81b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780b4d40 con 0x7f357815ee40 2023-11-04T05:29:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f357d81b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780c9d80 con 0x7f35780b4380 2023-11-04T05:29:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f35781f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f35781f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53912/0 (socket says 172.21.15.3:53912) 2023-11-04T05:29:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.433+0000 7f3576ffd700 1 -- 172.21.15.3:0/2110735481 learned_addr learned my addr 172.21.15.3:0/2110735481 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f35767fc700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f357815ee40 0x7f35780c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f35777fe700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35780b4380 0x7f35780b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f3576ffd700 1 -- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35780b4380 msgr2=0x7f35780b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f3576ffd700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35780b4380 0x7f35780b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f3576ffd700 1 -- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f357815ee40 msgr2=0x7f35780c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f3576ffd700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f357815ee40 0x7f35780c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f3576ffd700 1 -- 172.21.15.3:0/2110735481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357815e9b0 con 0x7f3560005210 2023-11-04T05:29:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f35767fc700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f357815ee40 0x7f35780c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f3576ffd700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f35781f2f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f356c0072c0 tx=0x7f356c0075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00f660 con 0x7f3560005210 2023-11-04T05:29:40.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c00f7c0 con 0x7f3560005210 2023-11-04T05:29:40.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35780c9470 con 0x7f3560005210 2023-11-04T05:29:40.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00fb70 con 0x7f3560005210 2023-11-04T05:29:40.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.434+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357815e360 con 0x7f3560005210 2023-11-04T05:29:40.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.435+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357815ec90 con 0x7f3560005210 2023-11-04T05:29:40.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.436+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f356c00fcd0 con 0x7f3560005210 2023-11-04T05:29:40.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.436+0000 7f355ffff700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f356406af00 0x7f356406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:40.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.436+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f356c088d10 con 0x7f3560005210 2023-11-04T05:29:40.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.436+0000 7f35767fc700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f356406af00 0x7f356406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:40.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.437+0000 7f35767fc700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f356406af00 0x7f356406d3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f357806d6a0 tx=0x7f3568008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:40.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.440+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f356c053fd0 con 0x7f3560005210 2023-11-04T05:29:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:40.644+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f35780d1030 con 0x7f3560005210 2023-11-04T05:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.221+0000 7f355ffff700 1 -- 172.21.15.3:0/2110735481 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f356c01a0b0 con 0x7f3560005210 2023-11-04T05:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.223+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f356406af00 msgr2=0x7f356406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.223+0000 7f357d81b700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f356406af00 0x7f356406d3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f357806d6a0 tx=0x7f3568008040 comp rx=0 tx=0).stop 2023-11-04T05:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.223+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 msgr2=0x7f35781f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.223+0000 7f357d81b700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f35781f2f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f356c0072c0 tx=0x7f356c0075d0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.223+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 shutdown_connections 2023-11-04T05:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.223+0000 7f357d81b700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f356406af00 0x7f356406d3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.224+0000 7f357d81b700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35780b4380 0x7f35780b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.224+0000 7f357d81b700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3560005210 0x7f35781f2f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.224+0000 7f357d81b700 1 --2- 172.21.15.3:0/2110735481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f357815ee40 0x7f35780c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.224+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 >> 172.21.15.3:0/2110735481 conn(0x7f357808aff0 msgr2=0x7f35780ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.224+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 shutdown_connections 2023-11-04T05:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.224+0000 7f357d81b700 1 -- 172.21.15.3:0/2110735481 wait complete. 2023-11-04T05:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-11-04T05:29:41.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-04T05:29:41.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.378+0000 7fd7667ad700 1 Processor -- start 2023-11-04T05:29:41.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.378+0000 7fd7667ad700 1 -- start start 2023-11-04T05:29:41.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd7667ad700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd76015dbd0 0x7fd7600b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd7667ad700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76015dfb0 con 0x7fd76006ab20 2023-11-04T05:29:41.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd7667ad700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7600b3070 con 0x7fd7600ccfc0 2023-11-04T05:29:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd7667ad700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7600b2780 con 0x7fd76015dbd0 2023-11-04T05:29:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd764d4a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd76006ab20 0x7fd76006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59272/0 (socket says 172.21.15.3:59272) 2023-11-04T05:29:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd764d4a700 1 -- 172.21.15.3:0/3169854396 learned_addr learned my addr 172.21.15.3:0/3169854396 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203842792 0 0) 0x7fd758001490 con 0x7fd76006ab20 2023-11-04T05:29:41.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.379+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74c0035b0 con 0x7fd76006ab20 2023-11-04T05:29:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75ffff700 1 --2- 172.21.15.3:0/3169854396 >> v2:172.21.15.188:3300/0 conn(0x7fd76015dbd0 0x7fd7600b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169042202 0 0) 0x7fd74c0035b0 con 0x7fd76006ab20 2023-11-04T05:29:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 >> v2:172.21.15.188:3300/0 conn(0x7fd76015dbd0 msgr2=0x7fd7600b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 --2- 172.21.15.3:0/3169854396 >> v2:172.21.15.188:3300/0 conn(0x7fd76015dbd0 0x7fd7600b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 >> v1:172.21.15.188:6789/0 conn(0x7fd7600ccfc0 legacy=0x7fd7600cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600cd510 con 0x7fd76006ab20 2023-11-04T05:29:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fd758001b80 con 0x7fd76006ab20 2023-11-04T05:29:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 >> v1:172.21.15.3:6789/0 conn(0x7fd76006ab20 legacy=0x7fd76006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd74c003a70 0x7fd74c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd74c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd74c005280 0x7fd74c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600b2780 con 0x7fd74c004850 2023-11-04T05:29:41.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c00c1e0 con 0x7fd74c003a70 2023-11-04T05:29:41.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c00c3a0 con 0x7fd74c005280 2023-11-04T05:29:41.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd74c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75ffff700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd74c003a70 0x7fd74c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd764d4a700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd74c005280 0x7fd74c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 -- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd74c005280 msgr2=0x7fd74c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:41.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd74c005280 0x7fd74c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 -- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd74c003a70 msgr2=0x7fd74c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:41.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd74c003a70 0x7fd74c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd764d4a700 1 -- 172.21.15.3:0/3169854396 reap_dead start 2023-11-04T05:29:41.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 -- 172.21.15.3:0/3169854396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c0035b0 con 0x7fd74c004850 2023-11-04T05:29:41.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.380+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd74c004cd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd750001620 tx=0x7fd74c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cda7610e1929c3a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:41.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500052d0 con 0x7fd74c004850 2023-11-04T05:29:41.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd750005450 con 0x7fd74c004850 2023-11-04T05:29:41.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd75effd700 1 -- 172.21.15.3:0/3169854396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500038c0 con 0x7fd74c004850 2023-11-04T05:29:41.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 -- 172.21.15.3:0/3169854396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 msgr2=0x7fd74c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd74c004cd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd750001620 tx=0x7fd74c005880 comp rx=0 tx=0).stop 2023-11-04T05:29:41.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 -- 172.21.15.3:0/3169854396 shutdown_connections 2023-11-04T05:29:41.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 --2- 172.21.15.3:0/3169854396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd74c004cd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 -- 172.21.15.3:0/3169854396 >> 172.21.15.3:0/3169854396 conn(0x7fd76006def0 msgr2=0x7fd76006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:41.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 -- 172.21.15.3:0/3169854396 shutdown_connections 2023-11-04T05:29:41.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.381+0000 7fd7667ad700 1 -- 172.21.15.3:0/3169854396 wait complete. 2023-11-04T05:29:41.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.382+0000 7fd7667ad700 1 Processor -- start 2023-11-04T05:29:41.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 -- start start 2023-11-04T05:29:41.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd76014a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76015fe40 0x7fd7601602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7600ac8e0 0x7fd7600acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76015a1a0 con 0x7fd74c004850 2023-11-04T05:29:41.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600ad2a0 con 0x7fd76015fe40 2023-11-04T05:29:41.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd7667ad700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76015eda0 con 0x7fd7600ac8e0 2023-11-04T05:29:41.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd76014a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd76014a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53932/0 (socket says 172.21.15.3:53932) 2023-11-04T05:29:41.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd75ffff700 1 -- 172.21.15.3:0/1957829037 learned_addr learned my addr 172.21.15.3:0/1957829037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:41.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd764d4a700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7600ac8e0 0x7fd7600acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.383+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76015fe40 0x7fd7601602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75ffff700 1 -- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7600ac8e0 msgr2=0x7fd7600acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75ffff700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7600ac8e0 0x7fd7600acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75ffff700 1 -- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76015fe40 msgr2=0x7fd7601602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:41.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75ffff700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76015fe40 0x7fd7601602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:41.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75ffff700 1 -- 172.21.15.3:0/1957829037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76015f080 con 0x7fd74c004850 2023-11-04T05:29:41.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75ffff700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd76014a9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd748011600 tx=0x7fd748011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:41.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74800c070 con 0x7fd74c004850 2023-11-04T05:29:41.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748015870 con 0x7fd74c004850 2023-11-04T05:29:41.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76015f360 con 0x7fd74c004850 2023-11-04T05:29:41.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748015c20 con 0x7fd74c004850 2023-11-04T05:29:41.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.384+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76015f920 con 0x7fd74c004850 2023-11-04T05:29:41.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.385+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7600b21b0 con 0x7fd74c004850 2023-11-04T05:29:41.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.386+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7480143e0 con 0x7fd74c004850 2023-11-04T05:29:41.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.386+0000 7fd75cff9700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd74006af30 0x7fd74006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:41.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.386+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fd748089560 con 0x7fd74c004850 2023-11-04T05:29:41.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.386+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd74006af30 0x7fd74006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:41.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.387+0000 7fd75f7fe700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd74006af30 0x7fd74006d3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd750000fd0 tx=0x7fd750009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:41.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.391+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd7480546b0 con 0x7fd74c004850 2023-11-04T05:29:41.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:41.593+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd76015c850 con 0x7fd74c004850 2023-11-04T05:29:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.227+0000 7fd75cff9700 1 -- 172.21.15.3:0/1957829037 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd748057cd0 con 0x7fd74c004850 2023-11-04T05:29:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd74006af30 msgr2=0x7fd74006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd74006af30 0x7fd74006d3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd750000fd0 tx=0x7fd750009030 comp rx=0 tx=0).stop 2023-11-04T05:29:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 msgr2=0x7fd76014a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd76014a9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd748011600 tx=0x7fd748011910 comp rx=0 tx=0).stop 2023-11-04T05:29:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 shutdown_connections 2023-11-04T05:29:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd74006af30 0x7fd74006d3f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7600ac8e0 0x7fd7600acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd74c004850 0x7fd76014a9c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 --2- 172.21.15.3:0/1957829037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76015fe40 0x7fd7601602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 >> 172.21.15.3:0/1957829037 conn(0x7fd76006def0 msgr2=0x7fd7600d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 shutdown_connections 2023-11-04T05:29:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.230+0000 7fd7667ad700 1 -- 172.21.15.3:0/1957829037 wait complete. 2023-11-04T05:29:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-11-04T05:29:42.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-04T05:29:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.390+0000 7f228a878700 1 Processor -- start 2023-11-04T05:29:42.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.390+0000 7f228a878700 1 -- start start 2023-11-04T05:29:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f228a878700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f228406ab20 0x7f228406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f228a878700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f22840d02d0 0x7f22840d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f228a878700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840caca0 con 0x7f22841611d0 2023-11-04T05:29:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f228a878700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22840ce0a0 con 0x7f22840d02d0 2023-11-04T05:29:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f228a878700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228406b4e0 con 0x7f228406ab20 2023-11-04T05:29:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f2288e15700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f22840d02d0 0x7f22840d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f2283fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f22841611d0 0x7f22840ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59288/0 (socket says 172.21.15.3:59288) 2023-11-04T05:29:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f2283fff700 1 -- 172.21.15.3:0/1192699123 learned_addr learned my addr 172.21.15.3:0/1192699123 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.391+0000 7f22837fe700 1 --2- 172.21.15.3:0/1192699123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f228406ab20 0x7f228406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2288e15700 1 -- 172.21.15.3:0/1192699123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f228406ab20 msgr2=0x7f228406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2288e15700 1 --2- 172.21.15.3:0/1192699123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f228406ab20 0x7f228406afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2288e15700 1 -- 172.21.15.3:0/1192699123 >> v1:172.21.15.3:6789/0 conn(0x7f22841611d0 legacy=0x7f22840ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2288e15700 1 -- 172.21.15.3:0/1192699123 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840cfe40 con 0x7f22840d02d0 2023-11-04T05:29:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2288e15700 1 --2- 172.21.15.3:0/1192699123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840d0760 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f227c009d90 tx=0x7f227c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be96e2294d5b6d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2282ffd700 1 -- 172.21.15.3:0/1192699123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00e070 con 0x7f22840d02d0 2023-11-04T05:29:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2282ffd700 1 -- 172.21.15.3:0/1192699123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c00bbd0 con 0x7f22840d02d0 2023-11-04T05:29:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f2282ffd700 1 -- 172.21.15.3:0/1192699123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c012430 con 0x7f22840d02d0 2023-11-04T05:29:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 -- 172.21.15.3:0/1192699123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 msgr2=0x7f22840d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 --2- 172.21.15.3:0/1192699123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840d0760 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f227c009d90 tx=0x7f227c00b330 comp rx=0 tx=0).stop 2023-11-04T05:29:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 -- 172.21.15.3:0/1192699123 shutdown_connections 2023-11-04T05:29:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 --2- 172.21.15.3:0/1192699123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840d0760 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 --2- 172.21.15.3:0/1192699123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f228406ab20 0x7f228406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 -- 172.21.15.3:0/1192699123 >> 172.21.15.3:0/1192699123 conn(0x7f2284072560 msgr2=0x7f2284099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.392+0000 7f228a878700 1 -- 172.21.15.3:0/1192699123 shutdown_connections 2023-11-04T05:29:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.393+0000 7f228a878700 1 -- 172.21.15.3:0/1192699123 wait complete. 2023-11-04T05:29:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.394+0000 7f228a878700 1 Processor -- start 2023-11-04T05:29:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.394+0000 7f228a878700 1 -- start start 2023-11-04T05:29:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228406ab20 0x7f228415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22841611d0 0x7f228415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841f7880 con 0x7f22840d02d0 2023-11-04T05:29:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840ca4e0 con 0x7f228406ab20 2023-11-04T05:29:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22841f6ac0 con 0x7f22841611d0 2023-11-04T05:29:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53948/0 (socket says 172.21.15.3:53948) 2023-11-04T05:29:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 -- 172.21.15.3:0/3446577955 learned_addr learned my addr 172.21.15.3:0/3446577955 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f2283fff700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228406ab20 0x7f228415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f2288e15700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22841611d0 0x7f228415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 -- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22841611d0 msgr2=0x7f228415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22841611d0 0x7f228415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 -- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228406ab20 msgr2=0x7f228415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228406ab20 0x7f228415af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 -- 172.21.15.3:0/3446577955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009a70 con 0x7f22840d02d0 2023-11-04T05:29:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f22837fe700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840c9be0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f227800e810 tx=0x7f227800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800c070 con 0x7f22840d02d0 2023-11-04T05:29:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278013610 con 0x7f22840d02d0 2023-11-04T05:29:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22841f6da0 con 0x7f22840d02d0 2023-11-04T05:29:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22780139c0 con 0x7f22840d02d0 2023-11-04T05:29:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.395+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22841f7360 con 0x7f22840d02d0 2023-11-04T05:29:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.396+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2284074df0 con 0x7f22840d02d0 2023-11-04T05:29:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.397+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2278013b20 con 0x7f22840d02d0 2023-11-04T05:29:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.397+0000 7f2280ff9700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af00 0x7f227006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.397+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f2278089410 con 0x7f22840d02d0 2023-11-04T05:29:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.398+0000 7f2283fff700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af00 0x7f227006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.398+0000 7f2283fff700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af00 0x7f227006d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2274005910 tx=0x7f2274009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.402+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f22780543f0 con 0x7f22840d02d0 2023-11-04T05:29:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:42.648+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f228406fca0 con 0x7f22840d02d0 2023-11-04T05:29:43.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.228+0000 7f2280ff9700 1 -- 172.21.15.3:0/3446577955 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2278057a10 con 0x7f22840d02d0 2023-11-04T05:29:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af00 msgr2=0x7f227006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af00 0x7f227006d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2274005910 tx=0x7f2274009040 comp rx=0 tx=0).stop 2023-11-04T05:29:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 msgr2=0x7f22840c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840c9be0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f227800e810 tx=0x7f227800eb20 comp rx=0 tx=0).stop 2023-11-04T05:29:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 shutdown_connections 2023-11-04T05:29:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f227006af00 0x7f227006d3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228406ab20 0x7f228415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22841611d0 0x7f228415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 --2- 172.21.15.3:0/3446577955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22840d02d0 0x7f22840c9be0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 >> 172.21.15.3:0/3446577955 conn(0x7f2284072560 msgr2=0x7f228406ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:43.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 shutdown_connections 2023-11-04T05:29:43.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.232+0000 7f228a878700 1 -- 172.21.15.3:0/3446577955 wait complete. 2023-11-04T05:29:43.237 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 5 and data pool 6 2023-11-04T05:29:43.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.377+0000 7fd1835aa700 1 Processor -- start 2023-11-04T05:29:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.378+0000 7fd1835aa700 1 -- start start 2023-11-04T05:29:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.378+0000 7fd1835aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 0x7fd17c0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.378+0000 7fd1835aa700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd17c071c00 0x7fd17c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.378+0000 7fd1835aa700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0720a0 con 0x7fd17c06dcd0 2023-11-04T05:29:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.378+0000 7fd1835aa700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c15e030 con 0x7fd17c071c00 2023-11-04T05:29:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.378+0000 7fd1835aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c06e140 con 0x7fd17c15dc50 2023-11-04T05:29:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd181b47700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd17c06dcd0 0x7fd17c06ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59322/0 (socket says 172.21.15.3:59322) 2023-11-04T05:29:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd181b47700 1 -- 172.21.15.3:0/1611547511 learned_addr learned my addr 172.21.15.3:0/1611547511 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd180b45700 1 --2- 172.21.15.3:0/1611547511 >> v2:172.21.15.188:3300/0 conn(0x7fd17c071c00 0x7fd17c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd181346700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 0x7fd17c0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195811880 0 0) 0x7fd17c0720a0 con 0x7fd17c06dcd0 2023-11-04T05:29:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1680035b0 con 0x7fd17c06dcd0 2023-11-04T05:29:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3980466982 0 0) 0x7fd1680035b0 con 0x7fd17c06dcd0 2023-11-04T05:29:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 msgr2=0x7fd17c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 0x7fd17c0716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 >> v2:172.21.15.188:3300/0 conn(0x7fd17c071c00 msgr2=0x7fd17c06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 --2- 172.21.15.3:0/1611547511 >> v2:172.21.15.188:3300/0 conn(0x7fd17c071c00 0x7fd17c06d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c072380 con 0x7fd17c06dcd0 2023-11-04T05:29:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fd1740019f0 con 0x7fd17c06dcd0 2023-11-04T05:29:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 >> v1:172.21.15.3:6789/0 conn(0x7fd17c06dcd0 legacy=0x7fd17c06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd168003a00 0x7fd168003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd168004850 0x7fd168004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 0x7fd168005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c15e030 con 0x7fd168003a00 2023-11-04T05:29:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c06e140 con 0x7fd168004850 2023-11-04T05:29:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800ba10 con 0x7fd17c15dc50 2023-11-04T05:29:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd181346700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd168003a00 0x7fd168003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd180b45700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd168004850 0x7fd168004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.379+0000 7fd181b47700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 0x7fd168005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181346700 1 -- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 msgr2=0x7fd168005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:43.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181346700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd17c15dc50 0x7fd168005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181346700 1 -- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd168004850 msgr2=0x7fd168004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:43.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181346700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd168004850 0x7fd168004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181346700 1 -- 172.21.15.3:0/1611547511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680035b0 con 0x7fd168003a00 2023-11-04T05:29:43.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181346700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd168003a00 0x7fd168003e20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009b60 tx=0x7fd168005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cb6c424d361e7e server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:43.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00e6c0 con 0x7fd168003a00 2023-11-04T05:29:43.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c00e820 con 0x7fd168003a00 2023-11-04T05:29:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd173fff700 1 -- 172.21.15.3:0/1611547511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00eb90 con 0x7fd168003a00 2023-11-04T05:29:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd1835aa700 1 -- 172.21.15.3:0/1611547511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd168003a00 msgr2=0x7fd168003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1611547511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd168003a00 0x7fd168003e20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009b60 tx=0x7fd168005880 comp rx=0 tx=0).stop 2023-11-04T05:29:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.380+0000 7fd181b47700 1 -- 172.21.15.3:0/1611547511 reap_dead start 2023-11-04T05:29:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.381+0000 7fd1835aa700 1 -- 172.21.15.3:0/1611547511 shutdown_connections 2023-11-04T05:29:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.381+0000 7fd1835aa700 1 -- 172.21.15.3:0/1611547511 >> 172.21.15.3:0/1611547511 conn(0x7fd17c0cfd50 msgr2=0x7fd17c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.381+0000 7fd1835aa700 1 -- 172.21.15.3:0/1611547511 shutdown_connections 2023-11-04T05:29:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.381+0000 7fd1835aa700 1 -- 172.21.15.3:0/1611547511 wait complete. 2023-11-04T05:29:43.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.382+0000 7fd1835aa700 1 Processor -- start 2023-11-04T05:29:43.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.382+0000 7fd1835aa700 1 -- start start 2023-11-04T05:29:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd1835aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd168003a00 0x7fd17c1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd1835aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0ad930 0x7fd17c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd1835aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 0x7fd17c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd1835aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0ae2f0 con 0x7fd17c0c7b80 2023-11-04T05:29:43.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd1835aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0c8540 con 0x7fd17c0ad930 2023-11-04T05:29:43.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd1835aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c15e280 con 0x7fd168003a00 2023-11-04T05:29:43.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 0x7fd17c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 0x7fd17c0c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53986/0 (socket says 172.21.15.3:53986) 2023-11-04T05:29:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 -- 172.21.15.3:0/1558440991 learned_addr learned my addr 172.21.15.3:0/1558440991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd180b45700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0ad930 0x7fd17c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181346700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd168003a00 0x7fd17c1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 -- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd168003a00 msgr2=0x7fd17c1f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd168003a00 0x7fd17c1f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 -- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0ad930 msgr2=0x7fd17c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0ad930 0x7fd17c0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 -- 172.21.15.3:0/1558440991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c15e560 con 0x7fd17c0c7b80 2023-11-04T05:29:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181346700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd168003a00 0x7fd17c1f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd181b47700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 0x7fd17c0c8000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0d0e50 tx=0x7fd17400a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.383+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740133f0 con 0x7fd17c0c7b80 2023-11-04T05:29:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.384+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174013550 con 0x7fd17c0c7b80 2023-11-04T05:29:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.384+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c15e840 con 0x7fd17c0c7b80 2023-11-04T05:29:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.384+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174013900 con 0x7fd17c0c7b80 2023-11-04T05:29:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.384+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c0c8760 con 0x7fd17c0c7b80 2023-11-04T05:29:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.385+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c15ed60 con 0x7fd17c0c7b80 2023-11-04T05:29:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.385+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd174013a60 con 0x7fd17c0c7b80 2023-11-04T05:29:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.385+0000 7fd171ffb700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd16006af00 0x7fd16006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.385+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd17408a0c0 con 0x7fd17c0c7b80 2023-11-04T05:29:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.386+0000 7fd181346700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd16006af00 0x7fd16006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.386+0000 7fd181346700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd16006af00 0x7fd16006d3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.390+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd17c15ed60 con 0x7fd17c0c7b80 2023-11-04T05:29:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.593+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd17c15ed60 con 0x7fd17c0c7b80 2023-11-04T05:29:43.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.594+0000 7fd171ffb700 1 -- 172.21.15.3:0/1558440991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16727 (secure 0 0 0) 0x7fd17c15ed60 con 0x7fd17c0c7b80 2023-11-04T05:29:43.597 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:43.597 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:42.650302+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:29:40.675722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-04T05:29:41.624805+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:43.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd16006af00 msgr2=0x7fd16006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd16006af00 0x7fd16006d3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c008030 comp rx=0 tx=0).stop 2023-11-04T05:29:43.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 msgr2=0x7fd17c0c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 0x7fd17c0c8000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0d0e50 tx=0x7fd17400a640 comp rx=0 tx=0).stop 2023-11-04T05:29:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 shutdown_connections 2023-11-04T05:29:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd16006af00 0x7fd16006d3c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd17c0c7b80 0x7fd17c0c8000 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd168003a00 0x7fd17c1f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 --2- 172.21.15.3:0/1558440991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd17c0ad930 0x7fd17c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 >> 172.21.15.3:0/1558440991 conn(0x7fd17c0cfd50 msgr2=0x7fd17c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 shutdown_connections 2023-11-04T05:29:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.597+0000 7fd1835aa700 1 -- 172.21.15.3:0/1558440991 wait complete. 2023-11-04T05:29:43.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:43.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.735+0000 7f26b2857700 1 Processor -- start 2023-11-04T05:29:43.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.735+0000 7f26b2857700 1 -- start start 2023-11-04T05:29:43.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b2857700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac1612c0 0x7f26ac06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b2857700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f26ac0b2e00 0x7f26ac0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b2857700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac1616a0 con 0x7f26ac0d1f80 2023-11-04T05:29:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b2857700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0b2bb0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b2857700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0d42c0 con 0x7f26ac1612c0 2023-11-04T05:29:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b0df4700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f26ac0d1f80 0x7f26ac0cf130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:37878/0 (socket says 172.21.15.3:37878) 2023-11-04T05:29:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26b0df4700 1 -- 172.21.15.3:0/1699915174 learned_addr learned my addr 172.21.15.3:0/1699915174 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26abfff700 1 --2- 172.21.15.3:0/1699915174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac1612c0 0x7f26ac06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.736+0000 7f26ab7fe700 1 --2- 172.21.15.3:0/1699915174 >> v2:172.21.15.188:3300/0 conn(0x7f26ac0b2e00 0x7f26ac0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26aaffd700 1 -- 172.21.15.3:0/1699915174 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22870548 0 0) 0x7f26ac1616a0 con 0x7f26ac0d1f80 2023-11-04T05:29:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26abfff700 1 -- 172.21.15.3:0/1699915174 >> v2:172.21.15.188:3300/0 conn(0x7f26ac0b2e00 msgr2=0x7f26ac0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26abfff700 1 --2- 172.21.15.3:0/1699915174 >> v2:172.21.15.188:3300/0 conn(0x7f26ac0b2e00 0x7f26ac0d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26abfff700 1 -- 172.21.15.3:0/1699915174 >> v1:172.21.15.188:6789/0 conn(0x7f26ac0d1f80 legacy=0x7f26ac0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26abfff700 1 -- 172.21.15.3:0/1699915174 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0d45a0 con 0x7f26ac1612c0 2023-11-04T05:29:43.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26abfff700 1 --2- 172.21.15.3:0/1699915174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac1612c0 0x7f26ac06bd30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f269c009820 tx=0x7f269c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93ad2ee84f12ebbb server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:43.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26aaffd700 1 -- 172.21.15.3:0/1699915174 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c005e30 con 0x7f26ac1612c0 2023-11-04T05:29:43.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26aaffd700 1 -- 172.21.15.3:0/1699915174 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c006030 con 0x7f26ac1612c0 2023-11-04T05:29:43.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.737+0000 7f26aaffd700 1 -- 172.21.15.3:0/1699915174 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0063c0 con 0x7f26ac1612c0 2023-11-04T05:29:43.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 -- 172.21.15.3:0/1699915174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac1612c0 msgr2=0x7f26ac06bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 --2- 172.21.15.3:0/1699915174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac1612c0 0x7f26ac06bd30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f269c009820 tx=0x7f269c009b30 comp rx=0 tx=0).stop 2023-11-04T05:29:43.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 -- 172.21.15.3:0/1699915174 shutdown_connections 2023-11-04T05:29:43.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 --2- 172.21.15.3:0/1699915174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac1612c0 0x7f26ac06bd30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 --2- 172.21.15.3:0/1699915174 >> v2:172.21.15.188:3300/0 conn(0x7f26ac0b2e00 0x7f26ac0d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 -- 172.21.15.3:0/1699915174 >> 172.21.15.3:0/1699915174 conn(0x7f26ac08aff0 msgr2=0x7f26ac0d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:43.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 -- 172.21.15.3:0/1699915174 shutdown_connections 2023-11-04T05:29:43.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.738+0000 7f26b2857700 1 -- 172.21.15.3:0/1699915174 wait complete. 2023-11-04T05:29:43.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.739+0000 7f26b2857700 1 Processor -- start 2023-11-04T05:29:43.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.739+0000 7f26b2857700 1 -- start start 2023-11-04T05:29:43.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b2857700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 0x7f26ac159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b2857700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac0d1f80 0x7f26ac0b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b2857700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac1612c0 0x7f26ac1f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b2857700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0cd090 con 0x7f26ac0b2e00 2023-11-04T05:29:43.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b2857700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0cd2f0 con 0x7f26ac1612c0 2023-11-04T05:29:43.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b2857700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0cd5d0 con 0x7f26ac0d1f80 2023-11-04T05:29:43.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 0x7f26ac159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 0x7f26ac159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53996/0 (socket says 172.21.15.3:53996) 2023-11-04T05:29:43.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 -- 172.21.15.3:0/1912127013 learned_addr learned my addr 172.21.15.3:0/1912127013 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:43.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26ab7fe700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac0d1f80 0x7f26ac0b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b0df4700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac1612c0 0x7f26ac1f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 -- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac0d1f80 msgr2=0x7f26ac0b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac0d1f80 0x7f26ac0b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 -- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac1612c0 msgr2=0x7f26ac1f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac1612c0 0x7f26ac1f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26abfff700 1 -- 172.21.15.3:0/1912127013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c009430 con 0x7f26ac0b2e00 2023-11-04T05:29:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.740+0000 7f26b0df4700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac1612c0 0x7f26ac1f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.741+0000 7f26abfff700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 0x7f26ac159f80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f269c000c00 tx=0x7f269c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.741+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0063a0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.741+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c020070 con 0x7f26ac0b2e00 2023-11-04T05:29:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.741+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c006740 con 0x7f26ac0b2e00 2023-11-04T05:29:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.741+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac0cd8b0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.741+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac15a4c0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.742+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac0c84d0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.742+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f269c022030 con 0x7f26ac0b2e00 2023-11-04T05:29:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.743+0000 7f26a8ff9700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f268c06af00 0x7f268c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.743+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f269c08ab80 con 0x7f26ac0b2e00 2023-11-04T05:29:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.743+0000 7f26ab7fe700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f268c06af00 0x7f268c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.743+0000 7f26ab7fe700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f268c06af00 0x7f268c06d3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2694009b40 tx=0x7f2694009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.747+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f26ac0c84d0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.950+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26ac0c84d0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.951+0000 7f26a8ff9700 1 -- 172.21.15.3:0/1912127013 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16714 (secure 0 0 0) 0x7f26ac0c84d0 con 0x7f26ac0b2e00 2023-11-04T05:29:43.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:43.954 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:43.832602+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:29:40.675722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-04T05:29:41.624805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:43.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.953+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f268c06af00 msgr2=0x7f268c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.953+0000 7f26b2857700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f268c06af00 0x7f268c06d3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2694009b40 tx=0x7f2694009210 comp rx=0 tx=0).stop 2023-11-04T05:29:43.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.953+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 msgr2=0x7f26ac159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.953+0000 7f26b2857700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 0x7f26ac159f80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f269c000c00 tx=0x7f269c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 shutdown_connections 2023-11-04T05:29:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f268c06af00 0x7f268c06d3c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac1612c0 0x7f26ac1f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ac0b2e00 0x7f26ac159f80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 --2- 172.21.15.3:0/1912127013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26ac0d1f80 0x7f26ac0b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 >> 172.21.15.3:0/1912127013 conn(0x7f26ac08aff0 msgr2=0x7f26ac06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 shutdown_connections 2023-11-04T05:29:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:43.954+0000 7f26b2857700 1 -- 172.21.15.3:0/1912127013 wait complete. 2023-11-04T05:29:43.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-04T05:29:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 Processor -- start 2023-11-04T05:29:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 -- start start 2023-11-04T05:29:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe35c15dd80 0x7fe35c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:44.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe35c0b3820 0x7fe35c161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c06d920 con 0x7fe35c1616e0 2023-11-04T05:29:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c162250 con 0x7fe35c15dd80 2023-11-04T05:29:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe3614d7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c06bee0 con 0x7fe35c0b3820 2023-11-04T05:29:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.096+0000 7fe35affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe35c15dd80 0x7fe35c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe35c15dd80 0x7fe35c06c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54004/0 (socket says 172.21.15.3:54004) 2023-11-04T05:29:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 -- 172.21.15.3:0/333580363 learned_addr learned my addr 172.21.15.3:0/333580363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35b7fe700 1 --2- 172.21.15.3:0/333580363 >> v2:172.21.15.188:3300/0 conn(0x7fe35c0b3820 0x7fe35c161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 -- 172.21.15.3:0/333580363 >> v2:172.21.15.188:3300/0 conn(0x7fe35c0b3820 msgr2=0x7fe35c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 --2- 172.21.15.3:0/333580363 >> v2:172.21.15.188:3300/0 conn(0x7fe35c0b3820 0x7fe35c161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 -- 172.21.15.3:0/333580363 >> v1:172.21.15.188:6789/0 conn(0x7fe35c1616e0 legacy=0x7fe35c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 -- 172.21.15.3:0/333580363 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c06c1c0 con 0x7fe35c15dd80 2023-11-04T05:29:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe35affd700 1 --2- 172.21.15.3:0/333580363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c15dd80 0x7fe35c06c9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0b10a0 tx=0x7fe34c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490890447e4e12d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe359ffb700 1 -- 172.21.15.3:0/333580363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c004230 con 0x7fe35c15dd80 2023-11-04T05:29:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe359ffb700 1 -- 172.21.15.3:0/333580363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c004390 con 0x7fe35c15dd80 2023-11-04T05:29:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.097+0000 7fe359ffb700 1 -- 172.21.15.3:0/333580363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0036a0 con 0x7fe35c15dd80 2023-11-04T05:29:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 -- 172.21.15.3:0/333580363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c15dd80 msgr2=0x7fe35c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 --2- 172.21.15.3:0/333580363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c15dd80 0x7fe35c06c9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0b10a0 tx=0x7fe34c009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 -- 172.21.15.3:0/333580363 shutdown_connections 2023-11-04T05:29:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 --2- 172.21.15.3:0/333580363 >> v2:172.21.15.188:3300/0 conn(0x7fe35c0b3820 0x7fe35c161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 --2- 172.21.15.3:0/333580363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c15dd80 0x7fe35c06c9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 -- 172.21.15.3:0/333580363 >> 172.21.15.3:0/333580363 conn(0x7fe35c0ce110 msgr2=0x7fe35c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 -- 172.21.15.3:0/333580363 shutdown_connections 2023-11-04T05:29:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.098+0000 7fe3614d7700 1 -- 172.21.15.3:0/333580363 wait complete. 2023-11-04T05:29:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.099+0000 7fe3614d7700 1 Processor -- start 2023-11-04T05:29:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.099+0000 7fe3614d7700 1 -- start start 2023-11-04T05:29:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe3614d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 0x7fe35c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe3614d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe35c1616e0 0x7fe35c0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe3614d7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35c0adad0 0x7fe35c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe3614d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c9ea0 con 0x7fe35c0b3820 2023-11-04T05:29:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe3614d7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0ca180 con 0x7fe35c0adad0 2023-11-04T05:29:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe3614d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0b4510 con 0x7fe35c1616e0 2023-11-04T05:29:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 0x7fe35c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 0x7fe35c151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54010/0 (socket says 172.21.15.3:54010) 2023-11-04T05:29:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 -- 172.21.15.3:0/1851352618 learned_addr learned my addr 172.21.15.3:0/1851352618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35b7fe700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35c0adad0 0x7fe35c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 -- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe35c1616e0 msgr2=0x7fe35c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35a7fc700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe35c1616e0 0x7fe35c0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe35c1616e0 0x7fe35c0caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 -- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35c0adad0 msgr2=0x7fe35c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35c0adad0 0x7fe35c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35affd700 1 -- 172.21.15.3:0/1851352618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c009550 con 0x7fe35c0b3820 2023-11-04T05:29:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.100+0000 7fe35b7fe700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35c0adad0 0x7fe35c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.101+0000 7fe35affd700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 0x7fe35c151ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe35c072890 tx=0x7fe34c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.101+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003dc0 con 0x7fe35c0b3820 2023-11-04T05:29:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.101+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c01f070 con 0x7fe35c0b3820 2023-11-04T05:29:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.101+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0b47f0 con 0x7fe35c0b3820 2023-11-04T05:29:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.101+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c011870 con 0x7fe35c0b3820 2023-11-04T05:29:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.101+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0cb910 con 0x7fe35c0b3820 2023-11-04T05:29:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.102+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c0b4d10 con 0x7fe35c0b3820 2023-11-04T05:29:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.102+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe34c021030 con 0x7fe35c0b3820 2023-11-04T05:29:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.103+0000 7fe3537fe700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe33c06af00 0x7fe33c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.103+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe34c08add0 con 0x7fe35c0b3820 2023-11-04T05:29:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.103+0000 7fe35a7fc700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe33c06af00 0x7fe33c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.104+0000 7fe35a7fc700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe33c06af00 0x7fe33c06d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0b0aa0 tx=0x7fe344008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.107+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fe34c055dd0 con 0x7fe35c0b3820 2023-11-04T05:29:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:44.358+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe35c074df0 con 0x7fe35c0b3820 2023-11-04T05:29:45.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.250+0000 7fe3537fe700 1 -- 172.21.15.3:0/1851352618 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe34c0593f0 con 0x7fe35c0b3820 2023-11-04T05:29:45.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe33c06af00 msgr2=0x7fe33c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:45.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe33c06af00 0x7fe33c06d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0b0aa0 tx=0x7fe344008040 comp rx=0 tx=0).stop 2023-11-04T05:29:45.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 msgr2=0x7fe35c151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:45.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 0x7fe35c151ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe35c072890 tx=0x7fe34c004750 comp rx=0 tx=0).stop 2023-11-04T05:29:45.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 shutdown_connections 2023-11-04T05:29:45.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fe33c06af00 0x7fe33c06d3c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35c0adad0 0x7fe35c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c0b3820 0x7fe35c151ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 --2- 172.21.15.3:0/1851352618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe35c1616e0 0x7fe35c0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 >> 172.21.15.3:0/1851352618 conn(0x7fe35c0ce110 msgr2=0x7fe35c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 shutdown_connections 2023-11-04T05:29:45.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.253+0000 7fe3614d7700 1 -- 172.21.15.3:0/1851352618 wait complete. 2023-11-04T05:29:45.271 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-04T05:29:45.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-04T05:29:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.405+0000 7fd688769700 1 Processor -- start 2023-11-04T05:29:45.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.405+0000 7fd688769700 1 -- start start 2023-11-04T05:29:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd688769700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd68015ba10 0x7fd6800c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd688769700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6800d0d70 0x7fd6800b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd688769700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd680070a90 con 0x7fd6800d11f0 2023-11-04T05:29:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd688769700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6800718f0 con 0x7fd6800d0d70 2023-11-04T05:29:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd688769700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800c5720 con 0x7fd68015ba10 2023-11-04T05:29:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd686505700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd68015ba10 0x7fd6800c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd685d04700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6800d0d70 0x7fd6800b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.406+0000 7fd686d06700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd6800d11f0 0x7fd68015f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.3:37908/0 (socket says 172.21.15.3:37908) 2023-11-04T05:29:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd686d06700 1 -- 172.21.15.3:0/2788662294 learned_addr learned my addr 172.21.15.3:0/2788662294 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685503700 1 -- 172.21.15.3:0/2788662294 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826042488 0 0) 0x7fd680070a90 con 0x7fd6800d11f0 2023-11-04T05:29:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685503700 1 -- 172.21.15.3:0/2788662294 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd66c0035b0 con 0x7fd6800d11f0 2023-11-04T05:29:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685d04700 1 -- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd68015ba10 msgr2=0x7fd6800c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685d04700 1 --2- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd68015ba10 0x7fd6800c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685d04700 1 -- 172.21.15.3:0/2788662294 >> v1:172.21.15.188:6789/0 conn(0x7fd6800d11f0 legacy=0x7fd68015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685d04700 1 -- 172.21.15.3:0/2788662294 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680070ce0 con 0x7fd6800d0d70 2023-11-04T05:29:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685d04700 1 --2- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6800d0d70 0x7fd6800b3110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009840 tx=0x7fd67c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482fbd23cb972767 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685503700 1 -- 172.21.15.3:0/2788662294 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c011830 con 0x7fd6800d0d70 2023-11-04T05:29:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685503700 1 -- 172.21.15.3:0/2788662294 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c011990 con 0x7fd6800d0d70 2023-11-04T05:29:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.407+0000 7fd685503700 1 -- 172.21.15.3:0/2788662294 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c011d20 con 0x7fd6800d0d70 2023-11-04T05:29:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 -- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6800d0d70 msgr2=0x7fd6800b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 --2- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6800d0d70 0x7fd6800b3110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009840 tx=0x7fd67c009b50 comp rx=0 tx=0).stop 2023-11-04T05:29:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 -- 172.21.15.3:0/2788662294 shutdown_connections 2023-11-04T05:29:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 --2- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd68015ba10 0x7fd6800c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 --2- 172.21.15.3:0/2788662294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6800d0d70 0x7fd6800b3110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 -- 172.21.15.3:0/2788662294 >> 172.21.15.3:0/2788662294 conn(0x7fd6800ce110 msgr2=0x7fd6800ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 -- 172.21.15.3:0/2788662294 shutdown_connections 2023-11-04T05:29:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.408+0000 7fd688769700 1 -- 172.21.15.3:0/2788662294 wait complete. 2023-11-04T05:29:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.409+0000 7fd688769700 1 Processor -- start 2023-11-04T05:29:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.409+0000 7fd688769700 1 -- start start 2023-11-04T05:29:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd688769700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6800d0d70 0x7fd6800afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd688769700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd68015ba10 0x7fd6800cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd688769700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 0x7fd6800cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd688769700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800cb370 con 0x7fd6800ca3c0 2023-11-04T05:29:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd688769700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6801f2500 con 0x7fd68015ba10 2023-11-04T05:29:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd688769700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6801f2220 con 0x7fd6800d0d70 2023-11-04T05:29:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 0x7fd6800cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 0x7fd6800cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54026/0 (socket says 172.21.15.3:54026) 2023-11-04T05:29:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 -- 172.21.15.3:0/3901828844 learned_addr learned my addr 172.21.15.3:0/3901828844 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd685d04700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd68015ba10 0x7fd6800cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686505700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6800d0d70 0x7fd6800afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 -- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6800d0d70 msgr2=0x7fd6800afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6800d0d70 0x7fd6800afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 -- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd68015ba10 msgr2=0x7fd6800cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd68015ba10 0x7fd6800cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.410+0000 7fd686d06700 1 -- 172.21.15.3:0/3901828844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c0094a0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.411+0000 7fd686d06700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 0x7fd6800cfcb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd67800cae0 tx=0x7fd67800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.411+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67800aac0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.411+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67800ac20 con 0x7fd6800ca3c0 2023-11-04T05:29:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.411+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6800c98e0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.411+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678012400 con 0x7fd6800ca3c0 2023-11-04T05:29:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.411+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6800c9ea0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.412+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68015ebe0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.412+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd67801b3b0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.413+0000 7fd676ffd700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd66406af00 0x7fd66406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.413+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd678089410 con 0x7fd6800ca3c0 2023-11-04T05:29:45.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.413+0000 7fd686505700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd66406af00 0x7fd66406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:45.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.413+0000 7fd686505700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd66406af00 0x7fd66406d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd670009a10 tx=0x7fd670008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:45.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.417+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd68015ebe0 con 0x7fd6800ca3c0 2023-11-04T05:29:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:45.666+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd68015ebe0 con 0x7fd6800ca3c0 2023-11-04T05:29:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.255+0000 7fd676ffd700 1 -- 172.21.15.3:0/3901828844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd68015ebe0 con 0x7fd6800ca3c0 2023-11-04T05:29:46.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd66406af00 msgr2=0x7fd66406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd66406af00 0x7fd66406d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd670009a10 tx=0x7fd670008040 comp rx=0 tx=0).stop 2023-11-04T05:29:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 msgr2=0x7fd6800cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 0x7fd6800cfcb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd67800cae0 tx=0x7fd67800cea0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 shutdown_connections 2023-11-04T05:29:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd66406af00 0x7fd66406d3c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6800ca3c0 0x7fd6800cfcb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6800d0d70 0x7fd6800afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 --2- 172.21.15.3:0/3901828844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd68015ba10 0x7fd6800cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 >> 172.21.15.3:0/3901828844 conn(0x7fd6800ce110 msgr2=0x7fd6800b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 shutdown_connections 2023-11-04T05:29:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.258+0000 7fd688769700 1 -- 172.21.15.3:0/3901828844 wait complete. 2023-11-04T05:29:46.275 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-04T05:29:46.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:29:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.406+0000 7f4fc84fb700 1 Processor -- start 2023-11-04T05:29:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.407+0000 7f4fc84fb700 1 -- start start 2023-11-04T05:29:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.407+0000 7f4fc84fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc00d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc84fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc006caf0 0x7f4fc0161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc84fb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc015df70 con 0x7f4fc00cd6d0 2023-11-04T05:29:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc84fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc006cf80 con 0x7f4fc006caf0 2023-11-04T05:29:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc84fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00d1690 con 0x7f4fc00cf230 2023-11-04T05:29:46.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc006caf0 0x7f4fc0161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc006caf0 0x7f4fc0161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54034/0 (socket says 172.21.15.3:54034) 2023-11-04T05:29:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 -- 172.21.15.3:0/598668354 learned_addr learned my addr 172.21.15.3:0/598668354 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6297700 1 --2- 172.21.15.3:0/598668354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc00d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 -- 172.21.15.3:0/598668354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 msgr2=0x7f4fc00d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 --2- 172.21.15.3:0/598668354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc00d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 -- 172.21.15.3:0/598668354 >> v1:172.21.15.188:6789/0 conn(0x7f4fc00cd6d0 legacy=0x7f4fc00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 -- 172.21.15.3:0/598668354 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc006d800 con 0x7f4fc006caf0 2023-11-04T05:29:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc6a98700 1 --2- 172.21.15.3:0/598668354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc0161640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009e50 tx=0x7f4fb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbaf1fe55e19e74 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc5295700 1 -- 172.21.15.3:0/598668354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800e070 con 0x7f4fc006caf0 2023-11-04T05:29:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc5295700 1 -- 172.21.15.3:0/598668354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb800b910 con 0x7f4fc006caf0 2023-11-04T05:29:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.408+0000 7f4fc5295700 1 -- 172.21.15.3:0/598668354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800bcc0 con 0x7f4fc006caf0 2023-11-04T05:29:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 -- 172.21.15.3:0/598668354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 msgr2=0x7f4fc0161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/598668354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc0161640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009e50 tx=0x7f4fb800b330 comp rx=0 tx=0).stop 2023-11-04T05:29:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 -- 172.21.15.3:0/598668354 shutdown_connections 2023-11-04T05:29:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/598668354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc0161640 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/598668354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc00d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 -- 172.21.15.3:0/598668354 >> 172.21.15.3:0/598668354 conn(0x7f4fc006ab20 msgr2=0x7f4fc006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 -- 172.21.15.3:0/598668354 shutdown_connections 2023-11-04T05:29:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.409+0000 7f4fc84fb700 1 -- 172.21.15.3:0/598668354 wait complete. 2023-11-04T05:29:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.410+0000 7f4fc84fb700 1 Processor -- start 2023-11-04T05:29:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.410+0000 7f4fc84fb700 1 -- start start 2023-11-04T05:29:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc84fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc84fb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00cd6d0 0x7f4fc01e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc84fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc01f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc84fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00b5cc0 con 0x7f4fc006caf0 2023-11-04T05:29:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc84fb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00b5700 con 0x7f4fc00cd6d0 2023-11-04T05:29:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc84fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00b59e0 con 0x7f4fc00cf230 2023-11-04T05:29:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc015b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54038/0 (socket says 172.21.15.3:54038) 2023-11-04T05:29:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 -- 172.21.15.3:0/1189498356 learned_addr learned my addr 172.21.15.3:0/1189498356 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 -- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 msgr2=0x7f4fc01f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc01f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 -- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00cd6d0 msgr2=0x7f4fc01e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00cd6d0 0x7f4fc01e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.411+0000 7f4fc6297700 1 -- 172.21.15.3:0/1189498356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8009b30 con 0x7f4fc006caf0 2023-11-04T05:29:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.412+0000 7f4fc6297700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc015b630 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e870 tx=0x7f4fb000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.412+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0009590 con 0x7f4fc006caf0 2023-11-04T05:29:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.412+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000ee70 con 0x7f4fc006caf0 2023-11-04T05:29:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.412+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00b6090 con 0x7f4fc006caf0 2023-11-04T05:29:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.412+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000ca50 con 0x7f4fc006caf0 2023-11-04T05:29:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.412+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc015bb70 con 0x7f4fc006caf0 2023-11-04T05:29:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.413+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc00b5eb0 con 0x7f4fc006caf0 2023-11-04T05:29:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.413+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fb000cbb0 con 0x7f4fc006caf0 2023-11-04T05:29:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.414+0000 7f4fb6ffd700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4fa406af00 0x7f4fa406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.414+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4fb00896b0 con 0x7f4fc006caf0 2023-11-04T05:29:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.414+0000 7f4fc5a96700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4fa406af00 0x7f4fa406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.415+0000 7f4fc5a96700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4fa406af00 0x7f4fa406d3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc000f60 tx=0x7f4fbc007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.419+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f4fb0054610 con 0x7f4fc006caf0 2023-11-04T05:29:46.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.665+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc00b33e0 con 0x7f4fc006caf0 2023-11-04T05:29:46.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.665+0000 7f4fb6ffd700 1 -- 172.21.15.3:0/1189498356 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4901 (secure 0 0 0) 0x7f4fb000ce60 con 0x7f4fc006caf0 2023-11-04T05:29:46.668 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:46.668 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2614810755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2614810755},{"type":"v1","addr":"172.21.15.188:6836","nonce":2614810755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5023,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/37625371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":37625371},{"type":"v1","addr":"172.21.15.188:6837","nonce":37625371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1141511123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1141511123},{"type":"v1","addr":"172.21.15.3:6834","nonce":1141511123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:29:42.650628+0000","modified":"2023-11-04T05:29:45.668525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/64074701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":64074701},{"type":"v1","addr":"172.21.15.3:6835","nonce":64074701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-04T05:29:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4fa406af00 msgr2=0x7f4fa406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4fa406af00 0x7f4fa406d3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc000f60 tx=0x7f4fbc007a40 comp rx=0 tx=0).stop 2023-11-04T05:29:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 msgr2=0x7f4fc015b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc015b630 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e870 tx=0x7f4fb000ec30 comp rx=0 tx=0).stop 2023-11-04T05:29:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 shutdown_connections 2023-11-04T05:29:46.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f4fa406af00 0x7f4fa406d3c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00cf230 0x7f4fc01f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc006caf0 0x7f4fc015b630 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 --2- 172.21.15.3:0/1189498356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00cd6d0 0x7f4fc01e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 >> 172.21.15.3:0/1189498356 conn(0x7f4fc006ab20 msgr2=0x7f4fc00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:46.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 shutdown_connections 2023-11-04T05:29:46.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.667+0000 7f4fc84fb700 1 -- 172.21.15.3:0/1189498356 wait complete. 2023-11-04T05:29:46.673 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2023-11-04T05:29:46.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T05:29:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.806+0000 7f09254b9700 1 Processor -- start 2023-11-04T05:29:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.806+0000 7f09254b9700 1 -- start start 2023-11-04T05:29:46.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.806+0000 7f09254b9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09201612c0 0x7f092006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f09254b9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09200b2e00 0x7f09200d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f09254b9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09201616a0 con 0x7f09200d1f80 2023-11-04T05:29:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f09254b9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200b2bb0 con 0x7f09200b2e00 2023-11-04T05:29:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f09254b9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200d42c0 con 0x7f09201612c0 2023-11-04T05:29:46.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091e7fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09200b2e00 0x7f09200d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f09200d1f80 0x7f09200cf130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59396/0 (socket says 172.21.15.3:59396) 2023-11-04T05:29:46.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091f7fe700 1 -- 172.21.15.3:0/1176544615 learned_addr learned my addr 172.21.15.3:0/1176544615 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:46.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091effd700 1 --2- 172.21.15.3:0/1176544615 >> v2:172.21.15.188:3300/0 conn(0x7f09201612c0 0x7f092006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091e7fc700 1 -- 172.21.15.3:0/1176544615 >> v2:172.21.15.188:3300/0 conn(0x7f09201612c0 msgr2=0x7f092006bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091e7fc700 1 --2- 172.21.15.3:0/1176544615 >> v2:172.21.15.188:3300/0 conn(0x7f09201612c0 0x7f092006bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091e7fc700 1 -- 172.21.15.3:0/1176544615 >> v1:172.21.15.3:6789/0 conn(0x7f09200d1f80 legacy=0x7f09200cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091e7fc700 1 -- 172.21.15.3:0/1176544615 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200d45a0 con 0x7f09200b2e00 2023-11-04T05:29:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091dffb700 1 -- 172.21.15.3:0/1176544615 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095527583 0 0) 0x7f09201616a0 con 0x7f09200d1f80 2023-11-04T05:29:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091e7fc700 1 --2- 172.21.15.3:0/1176544615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09200b2e00 0x7f09200d1780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f092008e0e0 tx=0x7f0914009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b440a6db338cde server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091dffb700 1 -- 172.21.15.3:0/1176544615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914011830 con 0x7f09200b2e00 2023-11-04T05:29:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091dffb700 1 -- 172.21.15.3:0/1176544615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914011990 con 0x7f09200b2e00 2023-11-04T05:29:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.807+0000 7f091dffb700 1 -- 172.21.15.3:0/1176544615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914011d20 con 0x7f09200b2e00 2023-11-04T05:29:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 -- 172.21.15.3:0/1176544615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09200b2e00 msgr2=0x7f09200d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 --2- 172.21.15.3:0/1176544615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09200b2e00 0x7f09200d1780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f092008e0e0 tx=0x7f0914009b20 comp rx=0 tx=0).stop 2023-11-04T05:29:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 -- 172.21.15.3:0/1176544615 shutdown_connections 2023-11-04T05:29:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 --2- 172.21.15.3:0/1176544615 >> v2:172.21.15.188:3300/0 conn(0x7f09201612c0 0x7f092006bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 --2- 172.21.15.3:0/1176544615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09200b2e00 0x7f09200d1780 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 -- 172.21.15.3:0/1176544615 >> 172.21.15.3:0/1176544615 conn(0x7f092008aff0 msgr2=0x7f09200d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 -- 172.21.15.3:0/1176544615 shutdown_connections 2023-11-04T05:29:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.808+0000 7f09254b9700 1 -- 172.21.15.3:0/1176544615 wait complete. 2023-11-04T05:29:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.809+0000 7f09254b9700 1 Processor -- start 2023-11-04T05:29:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.809+0000 7f09254b9700 1 -- start start 2023-11-04T05:29:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f09254b9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200b2e00 0x7f09201f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f09254b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09200d1f80 0x7f09201f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f09254b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 0x7f09201f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f09254b9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200b61a0 con 0x7f09201612c0 2023-11-04T05:29:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f09254b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200b5590 con 0x7f09200d1f80 2023-11-04T05:29:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f09254b9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200b5870 con 0x7f09200b2e00 2023-11-04T05:29:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f091f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 0x7f09201f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f091f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 0x7f09201f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54054/0 (socket says 172.21.15.3:54054) 2023-11-04T05:29:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f091f7fe700 1 -- 172.21.15.3:0/773683376 learned_addr learned my addr 172.21.15.3:0/773683376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f091e7fc700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09200d1f80 0x7f09201f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.810+0000 7f091effd700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200b2e00 0x7f09201f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f091f7fe700 1 -- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200b2e00 msgr2=0x7f09201f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f091f7fe700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200b2e00 0x7f09201f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f091f7fe700 1 -- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09200d1f80 msgr2=0x7f09201f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f091f7fe700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09200d1f80 0x7f09201f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f091f7fe700 1 -- 172.21.15.3:0/773683376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140094a0 con 0x7f09201612c0 2023-11-04T05:29:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f091f7fe700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 0x7f09201f6eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f091800cba0 tx=0x7f091800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800ab80 con 0x7f09201612c0 2023-11-04T05:29:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091800ad00 con 0x7f09201612c0 2023-11-04T05:29:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09200b5b50 con 0x7f09201612c0 2023-11-04T05:29:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918007310 con 0x7f09201612c0 2023-11-04T05:29:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.811+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092014d9f0 con 0x7f09201612c0 2023-11-04T05:29:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.812+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0920074df0 con 0x7f09201612c0 2023-11-04T05:29:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.812+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0918007470 con 0x7f09201612c0 2023-11-04T05:29:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.813+0000 7f09077fe700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f090c06af00 0x7f090c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.813+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0918089570 con 0x7f09201612c0 2023-11-04T05:29:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.813+0000 7f091effd700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f090c06af00 0x7f090c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.814+0000 7f091effd700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f090c06af00 0x7f090c06d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0910009ab0 tx=0x7f0910008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:46.817+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f09180544e0 con 0x7f09201612c0 2023-11-04T05:29:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.019+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09200d1d60 con 0x7f09201612c0 2023-11-04T05:29:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.021+0000 7f09077fe700 1 -- 172.21.15.3:0/773683376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16714 (secure 0 0 0) 0x7f0918057b00 con 0x7f09201612c0 2023-11-04T05:29:47.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:47.023 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T05:29:43.832602+0000","last_up_change":"2023-11-04T05:28:09.377140+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:29:40.675722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-04T05:29:41.624805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists","up"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T05:29:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f090c06af00 msgr2=0x7f090c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f090c06af00 0x7f090c06d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0910009ab0 tx=0x7f0910008040 comp rx=0 tx=0).stop 2023-11-04T05:29:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 msgr2=0x7f09201f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 0x7f09201f6eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f091800cba0 tx=0x7f091800cf60 comp rx=0 tx=0).stop 2023-11-04T05:29:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 shutdown_connections 2023-11-04T05:29:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f090c06af00 0x7f090c06d3c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09201612c0 0x7f09201f6eb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200b2e00 0x7f09201f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 --2- 172.21.15.3:0/773683376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09200d1f80 0x7f09201f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 >> 172.21.15.3:0/773683376 conn(0x7f092008aff0 msgr2=0x7f09200d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 shutdown_connections 2023-11-04T05:29:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.023+0000 7f09254b9700 1 -- 172.21.15.3:0/773683376 wait complete. 2023-11-04T05:29:47.037 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-04T05:29:47.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-04T05:29:47.093 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T05:29:47.091+0000 7fda4d42b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-04T05:29:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.161+0000 7f54cb57b700 1 Processor -- start 2023-11-04T05:29:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.161+0000 7f54cb57b700 1 -- start start 2023-11-04T05:29:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54cb57b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f54c40cc7a0 0x7f54c40728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54cb57b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54c40d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54cb57b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c4072df0 con 0x7f54c406ab20 2023-11-04T05:29:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54cb57b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c40cf050 con 0x7f54c40cc7a0 2023-11-04T05:29:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54cb57b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40cda00 con 0x7f54c406af00 2023-11-04T05:29:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54c9317700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f54c406ab20 0x7f54c40cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59406/0 (socket says 172.21.15.3:59406) 2023-11-04T05:29:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54c9317700 1 -- 172.21.15.3:0/2262110279 learned_addr learned my addr 172.21.15.3:0/2262110279 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54c8b16700 1 --2- 172.21.15.3:0/2262110279 >> v2:172.21.15.188:3300/0 conn(0x7f54c40cc7a0 0x7f54c40728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54c9b18700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54c40d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802384678 0 0) 0x7f54c4072df0 con 0x7f54c406ab20 2023-11-04T05:29:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54b00035b0 con 0x7f54c406ab20 2023-11-04T05:29:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2684848929 0 0) 0x7f54b00035b0 con 0x7f54c406ab20 2023-11-04T05:29:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 msgr2=0x7f54c40d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54c40d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 >> v2:172.21.15.188:3300/0 conn(0x7f54c40cc7a0 msgr2=0x7f54c40728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 --2- 172.21.15.3:0/2262110279 >> v2:172.21.15.188:3300/0 conn(0x7f54c40cc7a0 0x7f54c40728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c40cdce0 con 0x7f54c406ab20 2023-11-04T05:29:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f54c0001670 con 0x7f54c406ab20 2023-11-04T05:29:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54c9b18700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54c40d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 >> v1:172.21.15.3:6789/0 conn(0x7f54c406ab20 legacy=0x7f54c40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54b0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b0003f40 0x7f54b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0004d90 0x7f54b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40cf050 con 0x7f54b0003f40 2023-11-04T05:29:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40cda00 con 0x7f54b0004d90 2023-11-04T05:29:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.162+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b000ba40 con 0x7f54c406af00 2023-11-04T05:29:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b0003f40 0x7f54b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c9317700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54b0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c9b18700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0004d90 0x7f54b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 -- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 msgr2=0x7f54b0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c406af00 0x7f54b0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 -- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0004d90 msgr2=0x7f54b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0004d90 0x7f54b0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 -- 172.21.15.3:0/2262110279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00035b0 con 0x7f54b0003f40 2023-11-04T05:29:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54c8b16700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b0003f40 0x7f54b0004850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009b40 tx=0x7f54b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e97f0d6f8673dc server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4005a20 con 0x7f54b0003f40 2023-11-04T05:29:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4005b80 con 0x7f54b0003f40 2023-11-04T05:29:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.163+0000 7f54bbfff700 1 -- 172.21.15.3:0/2262110279 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400e6c0 con 0x7f54b0003f40 2023-11-04T05:29:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54cb57b700 1 -- 172.21.15.3:0/2262110279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b0003f40 msgr2=0x7f54b0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54cb57b700 1 --2- 172.21.15.3:0/2262110279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54b0003f40 0x7f54b0004850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009b40 tx=0x7f54b0005880 comp rx=0 tx=0).stop 2023-11-04T05:29:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54c9b18700 1 -- 172.21.15.3:0/2262110279 reap_dead start 2023-11-04T05:29:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54cb57b700 1 -- 172.21.15.3:0/2262110279 shutdown_connections 2023-11-04T05:29:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54cb57b700 1 -- 172.21.15.3:0/2262110279 >> 172.21.15.3:0/2262110279 conn(0x7f54c406df90 msgr2=0x7f54c406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54cb57b700 1 -- 172.21.15.3:0/2262110279 shutdown_connections 2023-11-04T05:29:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.164+0000 7f54cb57b700 1 -- 172.21.15.3:0/2262110279 wait complete. 2023-11-04T05:29:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.165+0000 7f54cb57b700 1 Processor -- start 2023-11-04T05:29:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.165+0000 7f54cb57b700 1 -- start start 2023-11-04T05:29:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54cb57b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b0003f40 0x7f54c415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54cb57b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 0x7f54c4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54cb57b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c4158c90 0x7f54c40ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54cb57b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4159140 con 0x7f54c4156ab0 2023-11-04T05:29:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54cb57b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c41577c0 con 0x7f54c4158c90 2023-11-04T05:29:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54cb57b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40ae600 con 0x7f54b0003f40 2023-11-04T05:29:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 0x7f54c4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 0x7f54c4158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54072/0 (socket says 172.21.15.3:54072) 2023-11-04T05:29:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 -- 172.21.15.3:0/438219255 learned_addr learned my addr 172.21.15.3:0/438219255 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 -- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b0003f40 msgr2=0x7f54c415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b0003f40 0x7f54c415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 -- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c4158c90 msgr2=0x7f54c40ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c9b18700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c4158c90 0x7f54c40ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c4158c90 0x7f54c40ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 -- 172.21.15.3:0/438219255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c40ae8e0 con 0x7f54c4156ab0 2023-11-04T05:29:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54c8b16700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 0x7f54c4158750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54b400ff40 tx=0x7f54b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.166+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4020070 con 0x7f54c4156ab0 2023-11-04T05:29:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.167+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4005e30 con 0x7f54c4156ab0 2023-11-04T05:29:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.167+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c40aebc0 con 0x7f54c4156ab0 2023-11-04T05:29:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.167+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4004480 con 0x7f54c4156ab0 2023-11-04T05:29:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.167+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c4156f30 con 0x7f54c4156ab0 2023-11-04T05:29:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.168+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c41e7320 con 0x7f54c4156ab0 2023-11-04T05:29:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.168+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54b40036a0 con 0x7f54c4156ab0 2023-11-04T05:29:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.168+0000 7f54b9ffb700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f54a806af00 0x7f54a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.169+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f54b408a920 con 0x7f54c4156ab0 2023-11-04T05:29:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.169+0000 7f54c9317700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f54a806af00 0x7f54a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.169+0000 7f54c9317700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f54a806af00 0x7f54a806d3c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54c40af3f0 tx=0x7f54c0000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.173+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f54b4055920 con 0x7f54c4156ab0 2023-11-04T05:29:47.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.372+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 --> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54c4072390 con 0x7f54a806af00 2023-11-04T05:29:47.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.381+0000 7f54b9ffb700 1 -- 172.21.15.3:0/438219255 <== mgr.4099 v2:172.21.15.188:6832/3102759174 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287637 (secure 0 0 0) 0x7f54c0001680 con 0x7f54a806af00 2023-11-04T05:29:47.387 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:47.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f54a806af00 msgr2=0x7f54a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f54a806af00 0x7f54a806d3c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f54c40af3f0 tx=0x7f54c0000bc0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 msgr2=0x7f54c4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 0x7f54c4158750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54b400ff40 tx=0x7f54b40047c0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 shutdown_connections 2023-11-04T05:29:47.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f54a806af00 0x7f54a806d3c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c4158c90 0x7f54c40ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b0003f40 0x7f54c415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 --2- 172.21.15.3:0/438219255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54c4156ab0 0x7f54c4158750 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 >> 172.21.15.3:0/438219255 conn(0x7f54c406df90 msgr2=0x7f54c40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:47.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 shutdown_connections 2023-11-04T05:29:47.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.384+0000 7f54cb57b700 1 -- 172.21.15.3:0/438219255 wait complete. 2023-11-04T05:29:47.392 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-11-04T05:29:47.399 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-11-04T05:29:45.773524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220876,"kb_used_data":4664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753844,"statfs":{"total":773094113280,"available":772867936256,"internally_reserved":0,"allocated":4775936,"data_stored":2696254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.549821"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.853183+0000","last_change":"2023-11-04T05:29:42.851926+0000","last_active":"2023-11-04T05:29:42.853183+0000","last_peered":"2023-11-04T05:29:42.853183+0000","last_clean":"2023-11-04T05:29:42.853183+0000","last_became_active":"2023-11-04T05:29:42.851407+0000","last_became_peered":"2023-11-04T05:29:42.851407+0000","last_unstale":"2023-11-04T05:29:42.853183+0000","last_undegraded":"2023-11-04T05:29:42.853183+0000","last_fullsized":"2023-11-04T05:29:42.853183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:38:32.412729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.858753+0000","last_change":"2023-11-04T05:29:42.857623+0000","last_active":"2023-11-04T05:29:42.858753+0000","last_peered":"2023-11-04T05:29:42.858753+0000","last_clean":"2023-11-04T05:29:42.858753+0000","last_became_active":"2023-11-04T05:29:42.857374+0000","last_became_peered":"2023-11-04T05:29:42.857374+0000","last_unstale":"2023-11-04T05:29:42.858753+0000","last_undegraded":"2023-11-04T05:29:42.858753+0000","last_fullsized":"2023-11-04T05:29:42.858753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:47:36.247887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856640+0000","last_change":"2023-11-04T05:29:42.854446+0000","last_active":"2023-11-04T05:29:42.856640+0000","last_peered":"2023-11-04T05:29:42.856640+0000","last_clean":"2023-11-04T05:29:42.856640+0000","last_became_active":"2023-11-04T05:29:42.854110+0000","last_became_peered":"2023-11-04T05:29:42.854110+0000","last_unstale":"2023-11-04T05:29:42.856640+0000","last_undegraded":"2023-11-04T05:29:42.856640+0000","last_fullsized":"2023-11-04T05:29:42.856640+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:02:46.294301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.857425+0000","last_change":"2023-11-04T05:29:42.852413+0000","last_active":"2023-11-04T05:29:42.857425+0000","last_peered":"2023-11-04T05:29:42.857425+0000","last_clean":"2023-11-04T05:29:42.857425+0000","last_became_active":"2023-11-04T05:29:42.852085+0000","last_became_peered":"2023-11-04T05:29:42.852085+0000","last_unstale":"2023-11-04T05:29:42.857425+0000","last_undegraded":"2023-11-04T05:29:42.857425+0000","last_fullsized":"2023-11-04T05:29:42.857425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:29:55.313459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856615+0000","last_change":"2023-11-04T05:29:42.850339+0000","last_active":"2023-11-04T05:29:42.856615+0000","last_peered":"2023-11-04T05:29:42.856615+0000","last_clean":"2023-11-04T05:29:42.856615+0000","last_became_active":"2023-11-04T05:29:42.850073+0000","last_became_peered":"2023-11-04T05:29:42.850073+0000","last_unstale":"2023-11-04T05:29:42.856615+0000","last_undegraded":"2023-11-04T05:29:42.856615+0000","last_fullsized":"2023-11-04T05:29:42.856615+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:37:39.180890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.850205+0000","last_change":"2023-11-04T05:29:42.848621+0000","last_active":"2023-11-04T05:29:42.850205+0000","last_peered":"2023-11-04T05:29:42.850205+0000","last_clean":"2023-11-04T05:29:42.850205+0000","last_became_active":"2023-11-04T05:29:42.848321+0000","last_became_peered":"2023-11-04T05:29:42.848321+0000","last_unstale":"2023-11-04T05:29:42.850205+0000","last_undegraded":"2023-11-04T05:29:42.850205+0000","last_fullsized":"2023-11-04T05:29:42.850205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:32.226284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044089+0000","last_change":"2023-11-04T05:29:42.852529+0000","last_active":"2023-11-04T05:29:44.044089+0000","last_peered":"2023-11-04T05:29:44.044089+0000","last_clean":"2023-11-04T05:29:44.044089+0000","last_became_active":"2023-11-04T05:29:42.852208+0000","last_became_peered":"2023-11-04T05:29:42.852208+0000","last_unstale":"2023-11-04T05:29:44.044089+0000","last_undegraded":"2023-11-04T05:29:44.044089+0000","last_fullsized":"2023-11-04T05:29:44.044089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:50:54.462544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851110+0000","last_change":"2023-11-04T05:29:42.847658+0000","last_active":"2023-11-04T05:29:42.851110+0000","last_peered":"2023-11-04T05:29:42.851110+0000","last_clean":"2023-11-04T05:29:42.851110+0000","last_became_active":"2023-11-04T05:29:42.847389+0000","last_became_peered":"2023-11-04T05:29:42.847389+0000","last_unstale":"2023-11-04T05:29:42.851110+0000","last_undegraded":"2023-11-04T05:29:42.851110+0000","last_fullsized":"2023-11-04T05:29:42.851110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:10:06.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.860512+0000","last_change":"2023-11-04T05:29:42.855975+0000","last_active":"2023-11-04T05:29:42.860512+0000","last_peered":"2023-11-04T05:29:42.860512+0000","last_clean":"2023-11-04T05:29:42.860512+0000","last_became_active":"2023-11-04T05:29:42.855662+0000","last_became_peered":"2023-11-04T05:29:42.855662+0000","last_unstale":"2023-11-04T05:29:42.860512+0000","last_undegraded":"2023-11-04T05:29:42.860512+0000","last_fullsized":"2023-11-04T05:29:42.860512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:44:41.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044162+0000","last_change":"2023-11-04T05:29:42.850320+0000","last_active":"2023-11-04T05:29:44.044162+0000","last_peered":"2023-11-04T05:29:44.044162+0000","last_clean":"2023-11-04T05:29:44.044162+0000","last_became_active":"2023-11-04T05:29:42.849982+0000","last_became_peered":"2023-11-04T05:29:42.849982+0000","last_unstale":"2023-11-04T05:29:44.044162+0000","last_undegraded":"2023-11-04T05:29:44.044162+0000","last_fullsized":"2023-11-04T05:29:44.044162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:42:04.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044354+0000","last_change":"2023-11-04T05:29:42.853690+0000","last_active":"2023-11-04T05:29:44.044354+0000","last_peered":"2023-11-04T05:29:44.044354+0000","last_clean":"2023-11-04T05:29:44.044354+0000","last_became_active":"2023-11-04T05:29:42.853255+0000","last_became_peered":"2023-11-04T05:29:42.853255+0000","last_unstale":"2023-11-04T05:29:44.044354+0000","last_undegraded":"2023-11-04T05:29:44.044354+0000","last_fullsized":"2023-11-04T05:29:44.044354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:49:16.749870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856346+0000","last_change":"2023-11-04T05:29:42.852973+0000","last_active":"2023-11-04T05:29:42.856346+0000","last_peered":"2023-11-04T05:29:42.856346+0000","last_clean":"2023-11-04T05:29:42.856346+0000","last_became_active":"2023-11-04T05:29:42.852680+0000","last_became_peered":"2023-11-04T05:29:42.852680+0000","last_unstale":"2023-11-04T05:29:42.856346+0000","last_undegraded":"2023-11-04T05:29:42.856346+0000","last_fullsized":"2023-11-04T05:29:42.856346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:32:03.063701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043919+0000","last_change":"2023-11-04T05:29:42.851462+0000","last_active":"2023-11-04T05:29:44.043919+0000","last_peered":"2023-11-04T05:29:44.043919+0000","last_clean":"2023-11-04T05:29:44.043919+0000","last_became_active":"2023-11-04T05:29:42.850955+0000","last_became_peered":"2023-11-04T05:29:42.850955+0000","last_unstale":"2023-11-04T05:29:44.043919+0000","last_undegraded":"2023-11-04T05:29:44.043919+0000","last_fullsized":"2023-11-04T05:29:44.043919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:33:47.033100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043784+0000","last_change":"2023-11-04T05:29:42.855140+0000","last_active":"2023-11-04T05:29:44.043784+0000","last_peered":"2023-11-04T05:29:44.043784+0000","last_clean":"2023-11-04T05:29:44.043784+0000","last_became_active":"2023-11-04T05:29:42.854854+0000","last_became_peered":"2023-11-04T05:29:42.854854+0000","last_unstale":"2023-11-04T05:29:44.043784+0000","last_undegraded":"2023-11-04T05:29:44.043784+0000","last_fullsized":"2023-11-04T05:29:44.043784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:52:31.106845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838737+0000","last_change":"2023-11-04T05:29:42.847281+0000","last_active":"2023-11-04T05:29:43.838737+0000","last_peered":"2023-11-04T05:29:43.838737+0000","last_clean":"2023-11-04T05:29:43.838737+0000","last_became_active":"2023-11-04T05:29:42.846333+0000","last_became_peered":"2023-11-04T05:29:42.846333+0000","last_unstale":"2023-11-04T05:29:43.838737+0000","last_undegraded":"2023-11-04T05:29:43.838737+0000","last_fullsized":"2023-11-04T05:29:43.838737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:43:13.164856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839470+0000","last_change":"2023-11-04T05:29:42.856256+0000","last_active":"2023-11-04T05:29:43.839470+0000","last_peered":"2023-11-04T05:29:43.839470+0000","last_clean":"2023-11-04T05:29:43.839470+0000","last_became_active":"2023-11-04T05:29:42.855935+0000","last_became_peered":"2023-11-04T05:29:42.855935+0000","last_unstale":"2023-11-04T05:29:43.839470+0000","last_undegraded":"2023-11-04T05:29:43.839470+0000","last_fullsized":"2023-11-04T05:29:43.839470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:47:36.247887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843568+0000","last_change":"2023-11-04T05:29:42.847691+0000","last_active":"2023-11-04T05:29:43.843568+0000","last_peered":"2023-11-04T05:29:43.843568+0000","last_clean":"2023-11-04T05:29:43.843568+0000","last_became_active":"2023-11-04T05:29:42.847352+0000","last_became_peered":"2023-11-04T05:29:42.847352+0000","last_unstale":"2023-11-04T05:29:43.843568+0000","last_undegraded":"2023-11-04T05:29:43.843568+0000","last_fullsized":"2023-11-04T05:29:43.843568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:40:17.203021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856584+0000","last_change":"2023-11-04T05:29:42.851801+0000","last_active":"2023-11-04T05:29:42.856584+0000","last_peered":"2023-11-04T05:29:42.856584+0000","last_clean":"2023-11-04T05:29:42.856584+0000","last_became_active":"2023-11-04T05:29:42.851547+0000","last_became_peered":"2023-11-04T05:29:42.851547+0000","last_unstale":"2023-11-04T05:29:42.856584+0000","last_undegraded":"2023-11-04T05:29:42.856584+0000","last_fullsized":"2023-11-04T05:29:42.856584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:56.746730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843418+0000","last_change":"2023-11-04T05:29:42.852379+0000","last_active":"2023-11-04T05:29:43.843418+0000","last_peered":"2023-11-04T05:29:43.843418+0000","last_clean":"2023-11-04T05:29:43.843418+0000","last_became_active":"2023-11-04T05:29:42.852096+0000","last_became_peered":"2023-11-04T05:29:42.852096+0000","last_unstale":"2023-11-04T05:29:43.843418+0000","last_undegraded":"2023-11-04T05:29:43.843418+0000","last_fullsized":"2023-11-04T05:29:43.843418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:34:33.323379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043953+0000","last_change":"2023-11-04T05:29:42.851264+0000","last_active":"2023-11-04T05:29:44.043953+0000","last_peered":"2023-11-04T05:29:44.043953+0000","last_clean":"2023-11-04T05:29:44.043953+0000","last_became_active":"2023-11-04T05:29:42.850608+0000","last_became_peered":"2023-11-04T05:29:42.850608+0000","last_unstale":"2023-11-04T05:29:44.043953+0000","last_undegraded":"2023-11-04T05:29:44.043953+0000","last_fullsized":"2023-11-04T05:29:44.043953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:31:35.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839994+0000","last_change":"2023-11-04T05:29:42.856316+0000","last_active":"2023-11-04T05:29:43.839994+0000","last_peered":"2023-11-04T05:29:43.839994+0000","last_clean":"2023-11-04T05:29:43.839994+0000","last_became_active":"2023-11-04T05:29:42.855696+0000","last_became_peered":"2023-11-04T05:29:42.855696+0000","last_unstale":"2023-11-04T05:29:43.839994+0000","last_undegraded":"2023-11-04T05:29:43.839994+0000","last_fullsized":"2023-11-04T05:29:43.839994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:31:35.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856732+0000","last_change":"2023-11-04T05:29:42.852388+0000","last_active":"2023-11-04T05:29:42.856732+0000","last_peered":"2023-11-04T05:29:42.856732+0000","last_clean":"2023-11-04T05:29:42.856732+0000","last_became_active":"2023-11-04T05:29:42.852078+0000","last_became_peered":"2023-11-04T05:29:42.852078+0000","last_unstale":"2023-11-04T05:29:42.856732+0000","last_undegraded":"2023-11-04T05:29:42.856732+0000","last_fullsized":"2023-11-04T05:29:42.856732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:23:19.733640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843618+0000","last_change":"2023-11-04T05:29:42.849330+0000","last_active":"2023-11-04T05:29:43.843618+0000","last_peered":"2023-11-04T05:29:43.843618+0000","last_clean":"2023-11-04T05:29:43.843618+0000","last_became_active":"2023-11-04T05:29:42.849017+0000","last_became_peered":"2023-11-04T05:29:42.849017+0000","last_unstale":"2023-11-04T05:29:43.843618+0000","last_undegraded":"2023-11-04T05:29:43.843618+0000","last_fullsized":"2023-11-04T05:29:43.843618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:28:10.802723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.860604+0000","last_change":"2023-11-04T05:29:42.856713+0000","last_active":"2023-11-04T05:29:42.860604+0000","last_peered":"2023-11-04T05:29:42.860604+0000","last_clean":"2023-11-04T05:29:42.860604+0000","last_became_active":"2023-11-04T05:29:42.856441+0000","last_became_peered":"2023-11-04T05:29:42.856441+0000","last_unstale":"2023-11-04T05:29:42.860604+0000","last_undegraded":"2023-11-04T05:29:42.860604+0000","last_fullsized":"2023-11-04T05:29:42.860604+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:28:10.802723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843608+0000","last_change":"2023-11-04T05:29:42.853519+0000","last_active":"2023-11-04T05:29:43.843608+0000","last_peered":"2023-11-04T05:29:43.843608+0000","last_clean":"2023-11-04T05:29:43.843608+0000","last_became_active":"2023-11-04T05:29:42.853233+0000","last_became_peered":"2023-11-04T05:29:42.853233+0000","last_unstale":"2023-11-04T05:29:43.843608+0000","last_undegraded":"2023-11-04T05:29:43.843608+0000","last_fullsized":"2023-11-04T05:29:43.843608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:30:24.898958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044136+0000","last_change":"2023-11-04T05:29:42.851273+0000","last_active":"2023-11-04T05:29:44.044136+0000","last_peered":"2023-11-04T05:29:44.044136+0000","last_clean":"2023-11-04T05:29:44.044136+0000","last_became_active":"2023-11-04T05:29:42.850985+0000","last_became_peered":"2023-11-04T05:29:42.850985+0000","last_unstale":"2023-11-04T05:29:44.044136+0000","last_undegraded":"2023-11-04T05:29:44.044136+0000","last_fullsized":"2023-11-04T05:29:44.044136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:57:23.290722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843386+0000","last_change":"2023-11-04T05:29:42.851236+0000","last_active":"2023-11-04T05:29:43.843386+0000","last_peered":"2023-11-04T05:29:43.843386+0000","last_clean":"2023-11-04T05:29:43.843386+0000","last_became_active":"2023-11-04T05:29:42.850943+0000","last_became_peered":"2023-11-04T05:29:42.850943+0000","last_unstale":"2023-11-04T05:29:43.843386+0000","last_undegraded":"2023-11-04T05:29:43.843386+0000","last_fullsized":"2023-11-04T05:29:43.843386+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:32.226284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843352+0000","last_change":"2023-11-04T05:29:42.846479+0000","last_active":"2023-11-04T05:29:43.843352+0000","last_peered":"2023-11-04T05:29:43.843352+0000","last_clean":"2023-11-04T05:29:43.843352+0000","last_became_active":"2023-11-04T05:29:42.846163+0000","last_became_peered":"2023-11-04T05:29:42.846163+0000","last_unstale":"2023-11-04T05:29:43.843352+0000","last_undegraded":"2023-11-04T05:29:43.843352+0000","last_fullsized":"2023-11-04T05:29:43.843352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:55:46.446134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839847+0000","last_change":"2023-11-04T05:29:42.855935+0000","last_active":"2023-11-04T05:29:43.839847+0000","last_peered":"2023-11-04T05:29:43.839847+0000","last_clean":"2023-11-04T05:29:43.839847+0000","last_became_active":"2023-11-04T05:29:42.853857+0000","last_became_peered":"2023-11-04T05:29:42.853857+0000","last_unstale":"2023-11-04T05:29:43.839847+0000","last_undegraded":"2023-11-04T05:29:43.839847+0000","last_fullsized":"2023-11-04T05:29:43.839847+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:38:32.412729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.857986+0000","last_change":"2023-11-04T05:29:42.855533+0000","last_active":"2023-11-04T05:29:42.857986+0000","last_peered":"2023-11-04T05:29:42.857986+0000","last_clean":"2023-11-04T05:29:42.857986+0000","last_became_active":"2023-11-04T05:29:42.855214+0000","last_became_peered":"2023-11-04T05:29:42.855214+0000","last_unstale":"2023-11-04T05:29:42.857986+0000","last_undegraded":"2023-11-04T05:29:42.857986+0000","last_fullsized":"2023-11-04T05:29:42.857986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:13:09.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839321+0000","last_change":"2023-11-04T05:29:42.856342+0000","last_active":"2023-11-04T05:29:43.839321+0000","last_peered":"2023-11-04T05:29:43.839321+0000","last_clean":"2023-11-04T05:29:43.839321+0000","last_became_active":"2023-11-04T05:29:42.856024+0000","last_became_peered":"2023-11-04T05:29:42.856024+0000","last_unstale":"2023-11-04T05:29:43.839321+0000","last_undegraded":"2023-11-04T05:29:43.839321+0000","last_fullsized":"2023-11-04T05:29:43.839321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:48:46.815700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843673+0000","last_change":"2023-11-04T05:29:42.244294+0000","last_active":"2023-11-04T05:29:43.843673+0000","last_peered":"2023-11-04T05:29:43.843673+0000","last_clean":"2023-11-04T05:29:43.843673+0000","last_became_active":"2023-11-04T05:29:42.243967+0000","last_became_peered":"2023-11-04T05:29:42.243967+0000","last_unstale":"2023-11-04T05:29:43.843673+0000","last_undegraded":"2023-11-04T05:29:43.843673+0000","last_fullsized":"2023-11-04T05:29:43.843673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:18:22.886731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.857952+0000","last_change":"2023-11-04T05:29:42.855431+0000","last_active":"2023-11-04T05:29:42.857952+0000","last_peered":"2023-11-04T05:29:42.857952+0000","last_clean":"2023-11-04T05:29:42.857952+0000","last_became_active":"2023-11-04T05:29:42.855062+0000","last_became_peered":"2023-11-04T05:29:42.855062+0000","last_unstale":"2023-11-04T05:29:42.857952+0000","last_undegraded":"2023-11-04T05:29:42.857952+0000","last_fullsized":"2023-11-04T05:29:42.857952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:49:44.251317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.242671+0000","last_change":"2023-11-04T05:29:42.250993+0000","last_active":"2023-11-04T05:29:43.242671+0000","last_peered":"2023-11-04T05:29:43.242671+0000","last_clean":"2023-11-04T05:29:43.242671+0000","last_became_active":"2023-11-04T05:29:42.249979+0000","last_became_peered":"2023-11-04T05:29:42.249979+0000","last_unstale":"2023-11-04T05:29:43.242671+0000","last_undegraded":"2023-11-04T05:29:43.242671+0000","last_fullsized":"2023-11-04T05:29:43.242671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:39:29.384933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.247051+0000","last_change":"2023-11-04T05:29:42.250623+0000","last_active":"2023-11-04T05:29:43.247051+0000","last_peered":"2023-11-04T05:29:43.247051+0000","last_clean":"2023-11-04T05:29:43.247051+0000","last_became_active":"2023-11-04T05:29:42.250258+0000","last_became_peered":"2023-11-04T05:29:42.250258+0000","last_unstale":"2023-11-04T05:29:43.247051+0000","last_undegraded":"2023-11-04T05:29:43.247051+0000","last_fullsized":"2023-11-04T05:29:43.247051+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:55:45.440618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856157+0000","last_change":"2023-11-04T05:29:42.853117+0000","last_active":"2023-11-04T05:29:42.856157+0000","last_peered":"2023-11-04T05:29:42.856157+0000","last_clean":"2023-11-04T05:29:42.856157+0000","last_became_active":"2023-11-04T05:29:42.852846+0000","last_became_peered":"2023-11-04T05:29:42.852846+0000","last_unstale":"2023-11-04T05:29:42.856157+0000","last_undegraded":"2023-11-04T05:29:42.856157+0000","last_fullsized":"2023-11-04T05:29:42.856157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:48:46.815700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-04T05:29:41.226427+0000","last_change":"2023-11-04T05:29:41.226427+0000","last_active":"2023-11-04T05:29:41.217438+0000","last_peered":"2023-11-04T05:29:41.217438+0000","last_clean":"2023-11-04T05:29:41.217438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:41.226427+0000","last_undegraded":"2023-11-04T05:29:41.226427+0000","last_fullsized":"2023-11-04T05:29:41.226427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.852133+0000","last_change":"2023-11-04T05:29:42.848581+0000","last_active":"2023-11-04T05:29:42.852133+0000","last_peered":"2023-11-04T05:29:42.852133+0000","last_clean":"2023-11-04T05:29:42.852133+0000","last_became_active":"2023-11-04T05:29:42.847752+0000","last_became_peered":"2023-11-04T05:29:42.847752+0000","last_unstale":"2023-11-04T05:29:42.852133+0000","last_undegraded":"2023-11-04T05:29:42.852133+0000","last_fullsized":"2023-11-04T05:29:42.852133+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:28:11.515210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044025+0000","last_change":"2023-11-04T05:29:42.256925+0000","last_active":"2023-11-04T05:29:44.044025+0000","last_peered":"2023-11-04T05:29:44.044025+0000","last_clean":"2023-11-04T05:29:44.044025+0000","last_became_active":"2023-11-04T05:29:42.256678+0000","last_became_peered":"2023-11-04T05:29:42.256678+0000","last_unstale":"2023-11-04T05:29:44.044025+0000","last_undegraded":"2023-11-04T05:29:44.044025+0000","last_fullsized":"2023-11-04T05:29:44.044025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:43:12.159340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043791+0000","last_change":"2023-11-04T05:29:42.855173+0000","last_active":"2023-11-04T05:29:44.043791+0000","last_peered":"2023-11-04T05:29:44.043791+0000","last_clean":"2023-11-04T05:29:44.043791+0000","last_became_active":"2023-11-04T05:29:42.854885+0000","last_became_peered":"2023-11-04T05:29:42.854885+0000","last_unstale":"2023-11-04T05:29:44.043791+0000","last_undegraded":"2023-11-04T05:29:44.043791+0000","last_fullsized":"2023-11-04T05:29:44.043791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:38:32.412729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838950+0000","last_change":"2023-11-04T05:29:42.251387+0000","last_active":"2023-11-04T05:29:43.838950+0000","last_peered":"2023-11-04T05:29:43.838950+0000","last_clean":"2023-11-04T05:29:43.838950+0000","last_became_active":"2023-11-04T05:29:42.251014+0000","last_became_peered":"2023-11-04T05:29:42.251014+0000","last_unstale":"2023-11-04T05:29:43.838950+0000","last_undegraded":"2023-11-04T05:29:43.838950+0000","last_fullsized":"2023-11-04T05:29:43.838950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:31.220768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.841805+0000","last_change":"2023-11-04T05:29:42.250268+0000","last_active":"2023-11-04T05:29:42.841805+0000","last_peered":"2023-11-04T05:29:42.841805+0000","last_clean":"2023-11-04T05:29:42.841805+0000","last_became_active":"2023-11-04T05:29:42.249994+0000","last_became_peered":"2023-11-04T05:29:42.249994+0000","last_unstale":"2023-11-04T05:29:42.841805+0000","last_undegraded":"2023-11-04T05:29:42.841805+0000","last_fullsized":"2023-11-04T05:29:42.841805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:34:32.317863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851449+0000","last_change":"2023-11-04T05:29:42.848295+0000","last_active":"2023-11-04T05:29:42.851449+0000","last_peered":"2023-11-04T05:29:42.851449+0000","last_clean":"2023-11-04T05:29:42.851449+0000","last_became_active":"2023-11-04T05:29:42.848039+0000","last_became_peered":"2023-11-04T05:29:42.848039+0000","last_unstale":"2023-11-04T05:29:42.851449+0000","last_undegraded":"2023-11-04T05:29:42.851449+0000","last_fullsized":"2023-11-04T05:29:42.851449+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:43:13.164856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-04T05:29:41.226196+0000","last_change":"2023-11-04T05:29:41.226196+0000","last_active":"2023-11-04T05:29:41.217438+0000","last_peered":"2023-11-04T05:29:41.217438+0000","last_clean":"2023-11-04T05:29:41.217438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:41.226196+0000","last_undegraded":"2023-11-04T05:29:41.226196+0000","last_fullsized":"2023-11-04T05:29:41.226196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856515+0000","last_change":"2023-11-04T05:29:42.851240+0000","last_active":"2023-11-04T05:29:42.856515+0000","last_peered":"2023-11-04T05:29:42.856515+0000","last_clean":"2023-11-04T05:29:42.856515+0000","last_became_active":"2023-11-04T05:29:42.850969+0000","last_became_peered":"2023-11-04T05:29:42.850969+0000","last_unstale":"2023-11-04T05:29:42.856515+0000","last_undegraded":"2023-11-04T05:29:42.856515+0000","last_fullsized":"2023-11-04T05:29:42.856515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:32:07.906761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.844672+0000","last_change":"2023-11-04T05:29:42.254224+0000","last_active":"2023-11-04T05:29:42.844672+0000","last_peered":"2023-11-04T05:29:42.844672+0000","last_clean":"2023-11-04T05:29:42.844672+0000","last_became_active":"2023-11-04T05:29:42.253917+0000","last_became_peered":"2023-11-04T05:29:42.253917+0000","last_unstale":"2023-11-04T05:29:42.844672+0000","last_undegraded":"2023-11-04T05:29:42.844672+0000","last_fullsized":"2023-11-04T05:29:42.844672+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:31:34.371973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839513+0000","last_change":"2023-11-04T05:29:42.851132+0000","last_active":"2023-11-04T05:29:43.839513+0000","last_peered":"2023-11-04T05:29:43.839513+0000","last_clean":"2023-11-04T05:29:43.839513+0000","last_became_active":"2023-11-04T05:29:42.850862+0000","last_became_peered":"2023-11-04T05:29:42.850862+0000","last_unstale":"2023-11-04T05:29:43.839513+0000","last_undegraded":"2023-11-04T05:29:43.839513+0000","last_fullsized":"2023-11-04T05:29:43.839513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:52:31.106845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044286+0000","last_change":"2023-11-04T05:29:42.245665+0000","last_active":"2023-11-04T05:29:44.044286+0000","last_peered":"2023-11-04T05:29:44.044286+0000","last_clean":"2023-11-04T05:29:44.044286+0000","last_became_active":"2023-11-04T05:29:42.245390+0000","last_became_peered":"2023-11-04T05:29:42.245390+0000","last_unstale":"2023-11-04T05:29:44.044286+0000","last_undegraded":"2023-11-04T05:29:44.044286+0000","last_fullsized":"2023-11-04T05:29:44.044286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:48:45.810184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.855647+0000","last_change":"2023-11-04T05:29:42.851283+0000","last_active":"2023-11-04T05:29:42.855647+0000","last_peered":"2023-11-04T05:29:42.855647+0000","last_clean":"2023-11-04T05:29:42.855647+0000","last_became_active":"2023-11-04T05:29:42.851033+0000","last_became_peered":"2023-11-04T05:29:42.851033+0000","last_unstale":"2023-11-04T05:29:42.855647+0000","last_undegraded":"2023-11-04T05:29:42.855647+0000","last_fullsized":"2023-11-04T05:29:42.855647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:20:03.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044421+0000","last_change":"2023-11-04T05:29:42.251479+0000","last_active":"2023-11-04T05:29:44.044421+0000","last_peered":"2023-11-04T05:29:44.044421+0000","last_clean":"2023-11-04T05:29:44.044421+0000","last_became_active":"2023-11-04T05:29:42.251178+0000","last_became_peered":"2023-11-04T05:29:42.251178+0000","last_unstale":"2023-11-04T05:29:44.044421+0000","last_undegraded":"2023-11-04T05:29:44.044421+0000","last_fullsized":"2023-11-04T05:29:44.044421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:03:56.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.245046+0000","last_change":"2023-11-04T05:29:42.246203+0000","last_active":"2023-11-04T05:29:43.245046+0000","last_peered":"2023-11-04T05:29:43.245046+0000","last_clean":"2023-11-04T05:29:43.245046+0000","last_became_active":"2023-11-04T05:29:42.245738+0000","last_became_peered":"2023-11-04T05:29:42.245738+0000","last_unstale":"2023-11-04T05:29:43.245046+0000","last_undegraded":"2023-11-04T05:29:43.245046+0000","last_fullsized":"2023-11-04T05:29:43.245046+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:33:46.027584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.855172+0000","last_change":"2023-11-04T05:29:42.851355+0000","last_active":"2023-11-04T05:29:42.855172+0000","last_peered":"2023-11-04T05:29:42.855172+0000","last_clean":"2023-11-04T05:29:42.855172+0000","last_became_active":"2023-11-04T05:29:42.850687+0000","last_became_peered":"2023-11-04T05:29:42.850687+0000","last_unstale":"2023-11-04T05:29:42.855172+0000","last_undegraded":"2023-11-04T05:29:42.855172+0000","last_fullsized":"2023-11-04T05:29:42.855172+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:53.484326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.240802+0000","last_change":"2023-11-04T05:29:42.251484+0000","last_active":"2023-11-04T05:29:43.240802+0000","last_peered":"2023-11-04T05:29:43.240802+0000","last_clean":"2023-11-04T05:29:43.240802+0000","last_became_active":"2023-11-04T05:29:42.251195+0000","last_became_peered":"2023-11-04T05:29:42.251195+0000","last_unstale":"2023-11-04T05:29:43.240802+0000","last_undegraded":"2023-11-04T05:29:43.240802+0000","last_fullsized":"2023-11-04T05:29:43.240802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:13:13.966265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.853190+0000","last_change":"2023-11-04T05:29:42.851604+0000","last_active":"2023-11-04T05:29:42.853190+0000","last_peered":"2023-11-04T05:29:42.853190+0000","last_clean":"2023-11-04T05:29:42.853190+0000","last_became_active":"2023-11-04T05:29:42.851144+0000","last_became_peered":"2023-11-04T05:29:42.851144+0000","last_unstale":"2023-11-04T05:29:42.853190+0000","last_undegraded":"2023-11-04T05:29:42.853190+0000","last_fullsized":"2023-11-04T05:29:42.853190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:47:27.907828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.842000+0000","last_change":"2023-11-04T05:29:42.249779+0000","last_active":"2023-11-04T05:29:42.842000+0000","last_peered":"2023-11-04T05:29:42.842000+0000","last_clean":"2023-11-04T05:29:42.842000+0000","last_became_active":"2023-11-04T05:29:42.249471+0000","last_became_peered":"2023-11-04T05:29:42.249471+0000","last_unstale":"2023-11-04T05:29:42.842000+0000","last_undegraded":"2023-11-04T05:29:42.842000+0000","last_fullsized":"2023-11-04T05:29:42.842000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:40:16.197505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838872+0000","last_change":"2023-11-04T05:29:42.854025+0000","last_active":"2023-11-04T05:29:43.838872+0000","last_peered":"2023-11-04T05:29:43.838872+0000","last_clean":"2023-11-04T05:29:43.838872+0000","last_became_active":"2023-11-04T05:29:42.853738+0000","last_became_peered":"2023-11-04T05:29:42.853738+0000","last_unstale":"2023-11-04T05:29:43.838872+0000","last_undegraded":"2023-11-04T05:29:43.838872+0000","last_fullsized":"2023-11-04T05:29:43.838872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:48:51.090986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843228+0000","last_change":"2023-11-04T05:29:42.244946+0000","last_active":"2023-11-04T05:29:43.843228+0000","last_peered":"2023-11-04T05:29:43.843228+0000","last_clean":"2023-11-04T05:29:43.843228+0000","last_became_active":"2023-11-04T05:29:42.244637+0000","last_became_peered":"2023-11-04T05:29:42.244637+0000","last_unstale":"2023-11-04T05:29:43.843228+0000","last_undegraded":"2023-11-04T05:29:43.843228+0000","last_fullsized":"2023-11-04T05:29:43.843228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:37:59.024785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856132+0000","last_change":"2023-11-04T05:29:42.853943+0000","last_active":"2023-11-04T05:29:42.856132+0000","last_peered":"2023-11-04T05:29:42.856132+0000","last_clean":"2023-11-04T05:29:42.856132+0000","last_became_active":"2023-11-04T05:29:42.853686+0000","last_became_peered":"2023-11-04T05:29:42.853686+0000","last_unstale":"2023-11-04T05:29:42.856132+0000","last_undegraded":"2023-11-04T05:29:42.856132+0000","last_fullsized":"2023-11-04T05:29:42.856132+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:03:57.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838835+0000","last_change":"2023-11-04T05:29:42.246692+0000","last_active":"2023-11-04T05:29:43.838835+0000","last_peered":"2023-11-04T05:29:43.838835+0000","last_clean":"2023-11-04T05:29:43.838835+0000","last_became_active":"2023-11-04T05:29:42.245957+0000","last_became_peered":"2023-11-04T05:29:42.245957+0000","last_unstale":"2023-11-04T05:29:43.838835+0000","last_undegraded":"2023-11-04T05:29:43.838835+0000","last_fullsized":"2023-11-04T05:29:43.838835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:52:48.500536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851058+0000","last_change":"2023-11-04T05:29:42.849176+0000","last_active":"2023-11-04T05:29:42.851058+0000","last_peered":"2023-11-04T05:29:42.851058+0000","last_clean":"2023-11-04T05:29:42.851058+0000","last_became_active":"2023-11-04T05:29:42.848947+0000","last_became_peered":"2023-11-04T05:29:42.848947+0000","last_unstale":"2023-11-04T05:29:42.851058+0000","last_undegraded":"2023-11-04T05:29:42.851058+0000","last_fullsized":"2023-11-04T05:29:42.851058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:51:25.063057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044383+0000","last_change":"2023-11-04T05:29:42.245183+0000","last_active":"2023-11-04T05:29:44.044383+0000","last_peered":"2023-11-04T05:29:44.044383+0000","last_clean":"2023-11-04T05:29:44.044383+0000","last_became_active":"2023-11-04T05:29:42.244897+0000","last_became_peered":"2023-11-04T05:29:42.244897+0000","last_unstale":"2023-11-04T05:29:44.044383+0000","last_undegraded":"2023-11-04T05:29:44.044383+0000","last_fullsized":"2023-11-04T05:29:44.044383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:48:50.085470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044356+0000","last_change":"2023-11-04T05:29:42.853518+0000","last_active":"2023-11-04T05:29:44.044356+0000","last_peered":"2023-11-04T05:29:44.044356+0000","last_clean":"2023-11-04T05:29:44.044356+0000","last_became_active":"2023-11-04T05:29:42.852829+0000","last_became_peered":"2023-11-04T05:29:42.852829+0000","last_unstale":"2023-11-04T05:29:44.044356+0000","last_undegraded":"2023-11-04T05:29:44.044356+0000","last_fullsized":"2023-11-04T05:29:44.044356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:11.307467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-04T05:29:41.226811+0000","last_change":"2023-11-04T05:29:41.226811+0000","last_active":"2023-11-04T05:29:41.217438+0000","last_peered":"2023-11-04T05:29:41.217438+0000","last_clean":"2023-11-04T05:29:41.217438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:41.226811+0000","last_undegraded":"2023-11-04T05:29:41.226811+0000","last_fullsized":"2023-11-04T05:29:41.226811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.241713+0000","last_change":"2023-11-04T05:29:42.249437+0000","last_active":"2023-11-04T05:29:43.241713+0000","last_peered":"2023-11-04T05:29:43.241713+0000","last_clean":"2023-11-04T05:29:43.241713+0000","last_became_active":"2023-11-04T05:29:42.249113+0000","last_became_peered":"2023-11-04T05:29:42.249113+0000","last_unstale":"2023-11-04T05:29:43.241713+0000","last_undegraded":"2023-11-04T05:29:43.241713+0000","last_fullsized":"2023-11-04T05:29:43.241713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:07:14.642689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856667+0000","last_change":"2023-11-04T05:29:42.852274+0000","last_active":"2023-11-04T05:29:42.856667+0000","last_peered":"2023-11-04T05:29:42.856667+0000","last_clean":"2023-11-04T05:29:42.856667+0000","last_became_active":"2023-11-04T05:29:42.852008+0000","last_became_peered":"2023-11-04T05:29:42.852008+0000","last_unstale":"2023-11-04T05:29:42.856667+0000","last_undegraded":"2023-11-04T05:29:42.856667+0000","last_fullsized":"2023-11-04T05:29:42.856667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:35:37.907113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.842367+0000","last_change":"2023-11-04T05:29:42.254132+0000","last_active":"2023-11-04T05:29:42.842367+0000","last_peered":"2023-11-04T05:29:42.842367+0000","last_clean":"2023-11-04T05:29:42.842367+0000","last_became_active":"2023-11-04T05:29:42.253781+0000","last_became_peered":"2023-11-04T05:29:42.253781+0000","last_unstale":"2023-11-04T05:29:42.842367+0000","last_undegraded":"2023-11-04T05:29:42.842367+0000","last_fullsized":"2023-11-04T05:29:42.842367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:30:23.893442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839470+0000","last_change":"2023-11-04T05:29:42.853230+0000","last_active":"2023-11-04T05:29:43.839470+0000","last_peered":"2023-11-04T05:29:43.839470+0000","last_clean":"2023-11-04T05:29:43.839470+0000","last_became_active":"2023-11-04T05:29:42.852935+0000","last_became_peered":"2023-11-04T05:29:42.852935+0000","last_unstale":"2023-11-04T05:29:43.839470+0000","last_undegraded":"2023-11-04T05:29:43.839470+0000","last_fullsized":"2023-11-04T05:29:43.839470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:27.494898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.240720+0000","last_change":"2023-11-04T05:29:42.256298+0000","last_active":"2023-11-04T05:29:43.240720+0000","last_peered":"2023-11-04T05:29:43.240720+0000","last_clean":"2023-11-04T05:29:43.240720+0000","last_became_active":"2023-11-04T05:29:42.255985+0000","last_became_peered":"2023-11-04T05:29:42.255985+0000","last_unstale":"2023-11-04T05:29:43.240720+0000","last_undegraded":"2023-11-04T05:29:43.240720+0000","last_fullsized":"2023-11-04T05:29:43.240720+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:49:23.763058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.855250+0000","last_change":"2023-11-04T05:29:42.850893+0000","last_active":"2023-11-04T05:29:42.855250+0000","last_peered":"2023-11-04T05:29:42.855250+0000","last_clean":"2023-11-04T05:29:42.855250+0000","last_became_active":"2023-11-04T05:29:42.850594+0000","last_became_peered":"2023-11-04T05:29:42.850594+0000","last_unstale":"2023-11-04T05:29:42.855250+0000","last_undegraded":"2023-11-04T05:29:42.855250+0000","last_fullsized":"2023-11-04T05:29:42.855250+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:33:43.907236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.843194+0000","last_change":"2023-11-04T05:29:42.246052+0000","last_active":"2023-11-04T05:29:42.843194+0000","last_peered":"2023-11-04T05:29:42.843194+0000","last_clean":"2023-11-04T05:29:42.843194+0000","last_became_active":"2023-11-04T05:29:42.245710+0000","last_became_peered":"2023-11-04T05:29:42.245710+0000","last_unstale":"2023-11-04T05:29:42.843194+0000","last_undegraded":"2023-11-04T05:29:42.843194+0000","last_fullsized":"2023-11-04T05:29:42.843194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:31:10.799253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043928+0000","last_change":"2023-11-04T05:29:42.851945+0000","last_active":"2023-11-04T05:29:44.043928+0000","last_peered":"2023-11-04T05:29:44.043928+0000","last_clean":"2023-11-04T05:29:44.043928+0000","last_became_active":"2023-11-04T05:29:42.851668+0000","last_became_peered":"2023-11-04T05:29:42.851668+0000","last_unstale":"2023-11-04T05:29:44.043928+0000","last_undegraded":"2023-11-04T05:29:44.043928+0000","last_fullsized":"2023-11-04T05:29:44.043928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:13:14.971781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.840052+0000","last_change":"2023-11-04T05:29:42.249736+0000","last_active":"2023-11-04T05:29:43.840052+0000","last_peered":"2023-11-04T05:29:43.840052+0000","last_clean":"2023-11-04T05:29:43.840052+0000","last_became_active":"2023-11-04T05:29:42.249345+0000","last_became_peered":"2023-11-04T05:29:42.249345+0000","last_unstale":"2023-11-04T05:29:43.840052+0000","last_undegraded":"2023-11-04T05:29:43.840052+0000","last_fullsized":"2023-11-04T05:29:43.840052+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:47:26.902313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851468+0000","last_change":"2023-11-04T05:29:42.848886+0000","last_active":"2023-11-04T05:29:42.851468+0000","last_peered":"2023-11-04T05:29:42.851468+0000","last_clean":"2023-11-04T05:29:42.851468+0000","last_became_active":"2023-11-04T05:29:42.848579+0000","last_became_peered":"2023-11-04T05:29:42.848579+0000","last_unstale":"2023-11-04T05:29:42.851468+0000","last_undegraded":"2023-11-04T05:29:42.851468+0000","last_fullsized":"2023-11-04T05:29:42.851468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:48:51.090986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.841401+0000","last_change":"2023-11-04T05:29:42.250049+0000","last_active":"2023-11-04T05:29:42.841401+0000","last_peered":"2023-11-04T05:29:42.841401+0000","last_clean":"2023-11-04T05:29:42.841401+0000","last_became_active":"2023-11-04T05:29:42.249781+0000","last_became_peered":"2023-11-04T05:29:42.249781+0000","last_unstale":"2023-11-04T05:29:42.841401+0000","last_undegraded":"2023-11-04T05:29:42.841401+0000","last_fullsized":"2023-11-04T05:29:42.841401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:54:56.672880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.858003+0000","last_change":"2023-11-04T05:29:42.856102+0000","last_active":"2023-11-04T05:29:42.858003+0000","last_peered":"2023-11-04T05:29:42.858003+0000","last_clean":"2023-11-04T05:29:42.858003+0000","last_became_active":"2023-11-04T05:29:42.855832+0000","last_became_peered":"2023-11-04T05:29:42.855832+0000","last_unstale":"2023-11-04T05:29:42.858003+0000","last_undegraded":"2023-11-04T05:29:42.858003+0000","last_fullsized":"2023-11-04T05:29:42.858003+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:35:50.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044444+0000","last_change":"2023-11-04T05:29:42.249712+0000","last_active":"2023-11-04T05:29:44.044444+0000","last_peered":"2023-11-04T05:29:44.044444+0000","last_clean":"2023-11-04T05:29:44.044444+0000","last_became_active":"2023-11-04T05:29:42.249435+0000","last_became_peered":"2023-11-04T05:29:42.249435+0000","last_unstale":"2023-11-04T05:29:44.044444+0000","last_undegraded":"2023-11-04T05:29:44.044444+0000","last_fullsized":"2023-11-04T05:29:44.044444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:26.489383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839858+0000","last_change":"2023-11-04T05:29:42.853782+0000","last_active":"2023-11-04T05:29:43.839858+0000","last_peered":"2023-11-04T05:29:43.839858+0000","last_clean":"2023-11-04T05:29:43.839858+0000","last_became_active":"2023-11-04T05:29:42.853476+0000","last_became_peered":"2023-11-04T05:29:42.853476+0000","last_unstale":"2023-11-04T05:29:43.839858+0000","last_undegraded":"2023-11-04T05:29:43.839858+0000","last_fullsized":"2023-11-04T05:29:43.839858+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:13:14.971781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.841609+0000","last_change":"2023-11-04T05:29:42.244493+0000","last_active":"2023-11-04T05:29:42.841609+0000","last_peered":"2023-11-04T05:29:42.841609+0000","last_clean":"2023-11-04T05:29:42.841609+0000","last_became_active":"2023-11-04T05:29:42.244176+0000","last_became_peered":"2023-11-04T05:29:42.244176+0000","last_unstale":"2023-11-04T05:29:42.841609+0000","last_undegraded":"2023-11-04T05:29:42.841609+0000","last_fullsized":"2023-11-04T05:29:42.841609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:39:40.090868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851054+0000","last_change":"2023-11-04T05:29:42.847667+0000","last_active":"2023-11-04T05:29:42.851054+0000","last_peered":"2023-11-04T05:29:42.851054+0000","last_clean":"2023-11-04T05:29:42.851054+0000","last_became_active":"2023-11-04T05:29:42.847385+0000","last_became_peered":"2023-11-04T05:29:42.847385+0000","last_unstale":"2023-11-04T05:29:42.851054+0000","last_undegraded":"2023-11-04T05:29:42.851054+0000","last_fullsized":"2023-11-04T05:29:42.851054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:27.494898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.839730+0000","last_change":"2023-11-04T05:29:42.251723+0000","last_active":"2023-11-04T05:29:42.839730+0000","last_peered":"2023-11-04T05:29:42.839730+0000","last_clean":"2023-11-04T05:29:42.839730+0000","last_became_active":"2023-11-04T05:29:42.251469+0000","last_became_peered":"2023-11-04T05:29:42.251469+0000","last_unstale":"2023-11-04T05:29:42.839730+0000","last_undegraded":"2023-11-04T05:29:42.839730+0000","last_fullsized":"2023-11-04T05:29:42.839730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:49:38.372915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.852111+0000","last_change":"2023-11-04T05:29:42.845934+0000","last_active":"2023-11-04T05:29:42.852111+0000","last_peered":"2023-11-04T05:29:42.852111+0000","last_clean":"2023-11-04T05:29:42.852111+0000","last_became_active":"2023-11-04T05:29:42.845654+0000","last_became_peered":"2023-11-04T05:29:42.845654+0000","last_unstale":"2023-11-04T05:29:42.852111+0000","last_undegraded":"2023-11-04T05:29:42.852111+0000","last_fullsized":"2023-11-04T05:29:42.852111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:56:20.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.840104+0000","last_change":"2023-11-04T05:29:42.253414+0000","last_active":"2023-11-04T05:29:42.840104+0000","last_peered":"2023-11-04T05:29:42.840104+0000","last_clean":"2023-11-04T05:29:42.840104+0000","last_became_active":"2023-11-04T05:29:42.253131+0000","last_became_peered":"2023-11-04T05:29:42.253131+0000","last_unstale":"2023-11-04T05:29:42.840104+0000","last_undegraded":"2023-11-04T05:29:42.840104+0000","last_fullsized":"2023-11-04T05:29:42.840104+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:18:16.548604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856392+0000","last_change":"2023-11-04T05:29:42.853402+0000","last_active":"2023-11-04T05:29:42.856392+0000","last_peered":"2023-11-04T05:29:42.856392+0000","last_clean":"2023-11-04T05:29:42.856392+0000","last_became_active":"2023-11-04T05:29:42.853145+0000","last_became_peered":"2023-11-04T05:29:42.853145+0000","last_unstale":"2023-11-04T05:29:42.856392+0000","last_undegraded":"2023-11-04T05:29:42.856392+0000","last_fullsized":"2023-11-04T05:29:42.856392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:18:23.892247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843308+0000","last_change":"2023-11-04T05:29:42.246860+0000","last_active":"2023-11-04T05:29:43.843308+0000","last_peered":"2023-11-04T05:29:43.843308+0000","last_clean":"2023-11-04T05:29:43.843308+0000","last_became_active":"2023-11-04T05:29:42.246588+0000","last_became_peered":"2023-11-04T05:29:42.246588+0000","last_unstale":"2023-11-04T05:29:43.843308+0000","last_undegraded":"2023-11-04T05:29:43.843308+0000","last_fullsized":"2023-11-04T05:29:43.843308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:49:43.245801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-04T05:29:42.222954+0000","last_change":"2023-11-04T05:29:42.222954+0000","last_active":"2023-11-04T05:29:42.222954+0000","last_peered":"2023-11-04T05:29:42.222954+0000","last_clean":"2023-11-04T05:29:42.222954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:42.222954+0000","last_undegraded":"2023-11-04T05:29:42.222954+0000","last_fullsized":"2023-11-04T05:29:42.222954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-04T05:29:41.227553+0000","last_change":"2023-11-04T05:29:41.227553+0000","last_active":"2023-11-04T05:29:41.217438+0000","last_peered":"2023-11-04T05:29:41.217438+0000","last_clean":"2023-11-04T05:29:41.217438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-04T05:29:41.227553+0000","last_undegraded":"2023-11-04T05:29:41.227553+0000","last_fullsized":"2023-11-04T05:29:41.227553+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839209+0000","last_change":"2023-11-04T05:29:42.845180+0000","last_active":"2023-11-04T05:29:43.839209+0000","last_peered":"2023-11-04T05:29:43.839209+0000","last_clean":"2023-11-04T05:29:43.839209+0000","last_became_active":"2023-11-04T05:29:42.844842+0000","last_became_peered":"2023-11-04T05:29:42.844842+0000","last_unstale":"2023-11-04T05:29:43.839209+0000","last_undegraded":"2023-11-04T05:29:43.839209+0000","last_fullsized":"2023-11-04T05:29:43.839209+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:03:57.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.843565+0000","last_change":"2023-11-04T05:29:42.248779+0000","last_active":"2023-11-04T05:29:42.843565+0000","last_peered":"2023-11-04T05:29:42.843565+0000","last_clean":"2023-11-04T05:29:42.843565+0000","last_became_active":"2023-11-04T05:29:42.248476+0000","last_became_peered":"2023-11-04T05:29:42.248476+0000","last_unstale":"2023-11-04T05:29:42.843565+0000","last_undegraded":"2023-11-04T05:29:42.843565+0000","last_fullsized":"2023-11-04T05:29:42.843565+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:52:30.101329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044306+0000","last_change":"2023-11-04T05:29:42.852568+0000","last_active":"2023-11-04T05:29:44.044306+0000","last_peered":"2023-11-04T05:29:44.044306+0000","last_clean":"2023-11-04T05:29:44.044306+0000","last_became_active":"2023-11-04T05:29:42.852246+0000","last_became_peered":"2023-11-04T05:29:42.852246+0000","last_unstale":"2023-11-04T05:29:44.044306+0000","last_undegraded":"2023-11-04T05:29:44.044306+0000","last_fullsized":"2023-11-04T05:29:44.044306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:58:39.441935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839399+0000","last_change":"2023-11-04T05:29:42.251343+0000","last_active":"2023-11-04T05:29:43.839399+0000","last_peered":"2023-11-04T05:29:43.839399+0000","last_clean":"2023-11-04T05:29:43.839399+0000","last_became_active":"2023-11-04T05:29:42.251040+0000","last_became_peered":"2023-11-04T05:29:42.251040+0000","last_unstale":"2023-11-04T05:29:43.839399+0000","last_undegraded":"2023-11-04T05:29:43.839399+0000","last_fullsized":"2023-11-04T05:29:43.839399+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:30:23.893442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.837201+0000","last_change":"2023-11-04T05:28:11.829985+0000","last_active":"2023-11-04T05:29:42.837201+0000","last_peered":"2023-11-04T05:29:42.837201+0000","last_clean":"2023-11-04T05:29:42.837201+0000","last_became_active":"2023-11-04T05:28:11.829705+0000","last_became_peered":"2023-11-04T05:28:11.829705+0000","last_unstale":"2023-11-04T05:29:42.837201+0000","last_undegraded":"2023-11-04T05:29:42.837201+0000","last_fullsized":"2023-11-04T05:29:42.837201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_clean_scrub_stamp":"2023-11-04T05:28:10.378947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:46:15.100605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1064960,"data_stored":787652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.8480000000000008,"5min":8.8480000000000008,"15min":8.8480000000000008},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.6020000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.7300000000000004,"5min":8.7300000000000004,"15min":8.7300000000000004},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.9410000000000007,"5min":8.9410000000000007,"15min":8.9410000000000007},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.7549999999999999,"5min":7.7549999999999999,"15min":7.7549999999999999},"last":0.73299999999999998}]},{"osd":3,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.0909999999999993,"5min":9.0909999999999993,"15min":9.0909999999999993},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.70399999999999996}]},{"osd":4,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.89900000000000002}]},{"osd":6,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71999999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":458752,"data_stored":196184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34499999999999997}]},{"osd":3,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.891,"5min":3.891,"15min":3.891},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.59299999999999997}]},{"osd":7,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.60999999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":779350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":11.079000000000001,"5min":11.079000000000001,"15min":11.079000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":10.798999999999999,"5min":10.798999999999999,"15min":10.798999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":11.958,"5min":11.958,"15min":11.958},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.100000000000001,"5min":20.100000000000001,"15min":20.100000000000001},"last":0.45700000000000002}]},{"osd":3,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":11.021000000000001,"5min":11.021000000000001,"15min":11.021000000000001},"last":0.433},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":15.153,"5min":15.153,"15min":15.153},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":10.939,"5min":10.939,"15min":10.939},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.99,"5min":10.99,"15min":10.99},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":10.879,"5min":10.879,"15min":10.879},"last":0.749},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":10.981999999999999,"5min":10.981999999999999,"15min":10.981999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":11.048,"5min":11.048,"15min":11.048},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":10.840999999999999,"5min":10.840999999999999,"15min":10.840999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":10.920999999999999,"5min":10.920999999999999,"15min":10.920999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":11.018000000000001,"5min":11.018000000000001,"15min":11.018000000000001},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":175292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":3.988,"5min":3.988,"15min":3.988},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":5.694,"5min":5.694,"15min":5.694},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":9.7789999999999999,"5min":9.7789999999999999,"15min":9.7789999999999999},"last":0.53800000000000003}]},{"osd":2,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.7729999999999997,"5min":5.7729999999999997,"15min":5.7729999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.8740000000000001,"5min":3.8740000000000001,"15min":3.8740000000000001},"last":0.498}]},{"osd":3,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.927,"5min":3.927,"15min":3.927},"last":0.58999999999999997}]},{"osd":5,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.57899999999999996}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.621}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60399999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":189072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.63}]},{"osd":4,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.755}]},{"osd":6,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81200000000000006}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.38800000000000001}]},{"osd":3,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63600000000000001}]},{"osd":5,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.624},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66200000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":190628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.51400000000000001}]},{"osd":2,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64000000000000001}]},{"osd":4,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.77200000000000002}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.87},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75600000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":189094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":8.0540000000000003,"5min":8.0540000000000003,"15min":8.0540000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":10.067,"5min":10.067,"15min":10.067},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":8.1300000000000008,"5min":8.1300000000000008,"15min":8.1300000000000008},"last":0.36699999999999999}]},{"osd":2,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":9.0890000000000004,"5min":9.0890000000000004,"15min":9.0890000000000004},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":10.169,"5min":10.169,"15min":10.169},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-04T05:29:47.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-04T05:29:47.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.562+0000 7fb1701bc700 1 Processor -- start 2023-11-04T05:29:47.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.562+0000 7fb1701bc700 1 -- start start 2023-11-04T05:29:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb1701bc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb1680728b0 0x7fb1681688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb1701bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680cda00 con 0x7fb16806ab20 2023-11-04T05:29:47.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb1701bc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680cdce0 con 0x7fb1680cc7a0 2023-11-04T05:29:47.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb1701bc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680cf050 con 0x7fb1680728b0 2023-11-04T05:29:47.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb16df58700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb16806ab20 0x7fb1680cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59426/0 (socket says 172.21.15.3:59426) 2023-11-04T05:29:47.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb16df58700 1 -- 172.21.15.3:0/4137778499 learned_addr learned my addr 172.21.15.3:0/4137778499 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:47.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb16e759700 1 --2- 172.21.15.3:0/4137778499 >> v2:172.21.15.188:3300/0 conn(0x7fb1680728b0 0x7fb1681688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.563+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743111547 0 0) 0x7fb1680cda00 con 0x7fb16806ab20 2023-11-04T05:29:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1540035b0 con 0x7fb16806ab20 2023-11-04T05:29:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939933075 0 0) 0x7fb1540035b0 con 0x7fb16806ab20 2023-11-04T05:29:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 >> v2:172.21.15.188:3300/0 conn(0x7fb1680728b0 msgr2=0x7fb1681688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 --2- 172.21.15.3:0/4137778499 >> v2:172.21.15.188:3300/0 conn(0x7fb1680728b0 0x7fb1681688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 >> v1:172.21.15.188:6789/0 conn(0x7fb1680cc7a0 legacy=0x7fb16806af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680720b0 con 0x7fb16806ab20 2023-11-04T05:29:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7fb1600019f0 con 0x7fb16806ab20 2023-11-04T05:29:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 >> v1:172.21.15.3:6789/0 conn(0x7fb16806ab20 legacy=0x7fb1680cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154003a00 0x7fb154003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb154004400 0x7fb1540048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb154005170 0x7fb15400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680cf050 con 0x7fb154005170 2023-11-04T05:29:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15400c0b0 con 0x7fb154003a00 2023-11-04T05:29:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15400c270 con 0x7fb154004400 2023-11-04T05:29:47.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb154005170 0x7fb15400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16df58700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154003a00 0x7fb154003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16d757700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb154004400 0x7fb1540048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 -- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb154004400 msgr2=0x7fb1540048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb154004400 0x7fb1540048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 -- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154003a00 msgr2=0x7fb154003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154003a00 0x7fb154003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 -- 172.21.15.3:0/4137778499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1540035b0 con 0x7fb154005170 2023-11-04T05:29:47.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 -- 172.21.15.3:0/4137778499 reap_dead start 2023-11-04T05:29:47.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.564+0000 7fb16e759700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb154005170 0x7fb15400ba10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb164009b60 tx=0x7fb154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635815c94a9dd53f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164004020 con 0x7fb154005170 2023-11-04T05:29:47.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb164004180 con 0x7fb154005170 2023-11-04T05:29:47.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb16cf56700 1 -- 172.21.15.3:0/4137778499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164004530 con 0x7fb154005170 2023-11-04T05:29:47.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 -- 172.21.15.3:0/4137778499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb154005170 msgr2=0x7fb15400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb154005170 0x7fb15400ba10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb164009b60 tx=0x7fb154005880 comp rx=0 tx=0).stop 2023-11-04T05:29:47.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 -- 172.21.15.3:0/4137778499 shutdown_connections 2023-11-04T05:29:47.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4137778499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb154005170 0x7fb15400ba10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 -- 172.21.15.3:0/4137778499 >> 172.21.15.3:0/4137778499 conn(0x7fb16806df90 msgr2=0x7fb16806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:47.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 -- 172.21.15.3:0/4137778499 shutdown_connections 2023-11-04T05:29:47.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.565+0000 7fb1701bc700 1 -- 172.21.15.3:0/4137778499 wait complete. 2023-11-04T05:29:47.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.566+0000 7fb1701bc700 1 Processor -- start 2023-11-04T05:29:47.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.566+0000 7fb1701bc700 1 -- start start 2023-11-04T05:29:47.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb1701bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154005170 0x7fb16815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb1701bc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb168155650 0x7fb1680c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb1701bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 0x7fb1680c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb1701bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681f4c00 con 0x7fb168155ad0 2023-11-04T05:29:47.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb1701bc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681f3f80 con 0x7fb154005170 2023-11-04T05:29:47.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb1701bc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681f4260 con 0x7fb168155650 2023-11-04T05:29:47.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb16e759700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 0x7fb1680c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb16e759700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 0x7fb1680c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54094/0 (socket says 172.21.15.3:54094) 2023-11-04T05:29:47.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb16e759700 1 -- 172.21.15.3:0/4050486353 learned_addr learned my addr 172.21.15.3:0/4050486353 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:47.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb16df58700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154005170 0x7fb16815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.567+0000 7fb16d757700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb168155650 0x7fb1680c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb16e759700 1 -- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb168155650 msgr2=0x7fb1680c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb16e759700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb168155650 0x7fb1680c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb16e759700 1 -- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154005170 msgr2=0x7fb16815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb16e759700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154005170 0x7fb16815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb16e759700 1 -- 172.21.15.3:0/4050486353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1681f4540 con 0x7fb168155ad0 2023-11-04T05:29:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb16e759700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 0x7fb1680c5f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1680ce1a0 tx=0x7fb1640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164004020 con 0x7fb168155ad0 2023-11-04T05:29:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1640038a0 con 0x7fb168155ad0 2023-11-04T05:29:47.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681f4820 con 0x7fb168155ad0 2023-11-04T05:29:47.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164003c50 con 0x7fb168155ad0 2023-11-04T05:29:47.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.568+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16814ed80 con 0x7fb168155ad0 2023-11-04T05:29:47.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.569+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16814efa0 con 0x7fb168155ad0 2023-11-04T05:29:47.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.569+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb164013470 con 0x7fb168155ad0 2023-11-04T05:29:47.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.570+0000 7fb15e7fc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb14c06af00 0x7fb14c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.570+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb16408a9f0 con 0x7fb168155ad0 2023-11-04T05:29:47.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.570+0000 7fb16df58700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb14c06af00 0x7fb14c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.570+0000 7fb16df58700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb14c06af00 0x7fb14c06d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb1680c5960 tx=0x7fb160006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.574+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fb16814efa0 con 0x7fb168155ad0 2023-11-04T05:29:47.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.773+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 --> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1681f43d0 con 0x7fb14c06af00 2023-11-04T05:29:47.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.782+0000 7fb15e7fc700 1 -- 172.21.15.3:0/4050486353 <== mgr.4099 v2:172.21.15.188:6832/3102759174 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289014 (secure 0 0 0) 0x7fb1681f43d0 con 0x7fb14c06af00 2023-11-04T05:29:47.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:47.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb14c06af00 msgr2=0x7fb14c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb14c06af00 0x7fb14c06d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb1680c5960 tx=0x7fb160006040 comp rx=0 tx=0).stop 2023-11-04T05:29:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 msgr2=0x7fb1680c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 0x7fb1680c5f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1680ce1a0 tx=0x7fb1640047c0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 shutdown_connections 2023-11-04T05:29:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fb14c06af00 0x7fb14c06d3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb154005170 0x7fb16815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb168155ad0 0x7fb1680c5f00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 --2- 172.21.15.3:0/4050486353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb168155650 0x7fb1680c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 >> 172.21.15.3:0/4050486353 conn(0x7fb16806df90 msgr2=0x7fb16815fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 shutdown_connections 2023-11-04T05:29:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.785+0000 7fb1701bc700 1 -- 172.21.15.3:0/4050486353 wait complete. 2023-11-04T05:29:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-11-04T05:29:47.800 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-11-04T05:29:47.774022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220924,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753796,"statfs":{"total":773094113280,"available":772867887104,"internally_reserved":0,"allocated":4820992,"data_stored":2717168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001712"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.853183+0000","last_change":"2023-11-04T05:29:42.851926+0000","last_active":"2023-11-04T05:29:42.853183+0000","last_peered":"2023-11-04T05:29:42.853183+0000","last_clean":"2023-11-04T05:29:42.853183+0000","last_became_active":"2023-11-04T05:29:42.851407+0000","last_became_peered":"2023-11-04T05:29:42.851407+0000","last_unstale":"2023-11-04T05:29:42.853183+0000","last_undegraded":"2023-11-04T05:29:42.853183+0000","last_fullsized":"2023-11-04T05:29:42.853183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:38:32.412729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.858753+0000","last_change":"2023-11-04T05:29:42.857623+0000","last_active":"2023-11-04T05:29:42.858753+0000","last_peered":"2023-11-04T05:29:42.858753+0000","last_clean":"2023-11-04T05:29:42.858753+0000","last_became_active":"2023-11-04T05:29:42.857374+0000","last_became_peered":"2023-11-04T05:29:42.857374+0000","last_unstale":"2023-11-04T05:29:42.858753+0000","last_undegraded":"2023-11-04T05:29:42.858753+0000","last_fullsized":"2023-11-04T05:29:42.858753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:47:36.247887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856640+0000","last_change":"2023-11-04T05:29:42.854446+0000","last_active":"2023-11-04T05:29:42.856640+0000","last_peered":"2023-11-04T05:29:42.856640+0000","last_clean":"2023-11-04T05:29:42.856640+0000","last_became_active":"2023-11-04T05:29:42.854110+0000","last_became_peered":"2023-11-04T05:29:42.854110+0000","last_unstale":"2023-11-04T05:29:42.856640+0000","last_undegraded":"2023-11-04T05:29:42.856640+0000","last_fullsized":"2023-11-04T05:29:42.856640+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:02:46.294301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.857425+0000","last_change":"2023-11-04T05:29:42.852413+0000","last_active":"2023-11-04T05:29:42.857425+0000","last_peered":"2023-11-04T05:29:42.857425+0000","last_clean":"2023-11-04T05:29:42.857425+0000","last_became_active":"2023-11-04T05:29:42.852085+0000","last_became_peered":"2023-11-04T05:29:42.852085+0000","last_unstale":"2023-11-04T05:29:42.857425+0000","last_undegraded":"2023-11-04T05:29:42.857425+0000","last_fullsized":"2023-11-04T05:29:42.857425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:29:55.313459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856615+0000","last_change":"2023-11-04T05:29:42.850339+0000","last_active":"2023-11-04T05:29:42.856615+0000","last_peered":"2023-11-04T05:29:42.856615+0000","last_clean":"2023-11-04T05:29:42.856615+0000","last_became_active":"2023-11-04T05:29:42.850073+0000","last_became_peered":"2023-11-04T05:29:42.850073+0000","last_unstale":"2023-11-04T05:29:42.856615+0000","last_undegraded":"2023-11-04T05:29:42.856615+0000","last_fullsized":"2023-11-04T05:29:42.856615+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:37:39.180890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.850205+0000","last_change":"2023-11-04T05:29:42.848621+0000","last_active":"2023-11-04T05:29:42.850205+0000","last_peered":"2023-11-04T05:29:42.850205+0000","last_clean":"2023-11-04T05:29:42.850205+0000","last_became_active":"2023-11-04T05:29:42.848321+0000","last_became_peered":"2023-11-04T05:29:42.848321+0000","last_unstale":"2023-11-04T05:29:42.850205+0000","last_undegraded":"2023-11-04T05:29:42.850205+0000","last_fullsized":"2023-11-04T05:29:42.850205+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:32.226284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044089+0000","last_change":"2023-11-04T05:29:42.852529+0000","last_active":"2023-11-04T05:29:44.044089+0000","last_peered":"2023-11-04T05:29:44.044089+0000","last_clean":"2023-11-04T05:29:44.044089+0000","last_became_active":"2023-11-04T05:29:42.852208+0000","last_became_peered":"2023-11-04T05:29:42.852208+0000","last_unstale":"2023-11-04T05:29:44.044089+0000","last_undegraded":"2023-11-04T05:29:44.044089+0000","last_fullsized":"2023-11-04T05:29:44.044089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:50:54.462544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851110+0000","last_change":"2023-11-04T05:29:42.847658+0000","last_active":"2023-11-04T05:29:42.851110+0000","last_peered":"2023-11-04T05:29:42.851110+0000","last_clean":"2023-11-04T05:29:42.851110+0000","last_became_active":"2023-11-04T05:29:42.847389+0000","last_became_peered":"2023-11-04T05:29:42.847389+0000","last_unstale":"2023-11-04T05:29:42.851110+0000","last_undegraded":"2023-11-04T05:29:42.851110+0000","last_fullsized":"2023-11-04T05:29:42.851110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:10:06.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.860512+0000","last_change":"2023-11-04T05:29:42.855975+0000","last_active":"2023-11-04T05:29:42.860512+0000","last_peered":"2023-11-04T05:29:42.860512+0000","last_clean":"2023-11-04T05:29:42.860512+0000","last_became_active":"2023-11-04T05:29:42.855662+0000","last_became_peered":"2023-11-04T05:29:42.855662+0000","last_unstale":"2023-11-04T05:29:42.860512+0000","last_undegraded":"2023-11-04T05:29:42.860512+0000","last_fullsized":"2023-11-04T05:29:42.860512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:44:41.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044162+0000","last_change":"2023-11-04T05:29:42.850320+0000","last_active":"2023-11-04T05:29:44.044162+0000","last_peered":"2023-11-04T05:29:44.044162+0000","last_clean":"2023-11-04T05:29:44.044162+0000","last_became_active":"2023-11-04T05:29:42.849982+0000","last_became_peered":"2023-11-04T05:29:42.849982+0000","last_unstale":"2023-11-04T05:29:44.044162+0000","last_undegraded":"2023-11-04T05:29:44.044162+0000","last_fullsized":"2023-11-04T05:29:44.044162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:42:04.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044354+0000","last_change":"2023-11-04T05:29:42.853690+0000","last_active":"2023-11-04T05:29:44.044354+0000","last_peered":"2023-11-04T05:29:44.044354+0000","last_clean":"2023-11-04T05:29:44.044354+0000","last_became_active":"2023-11-04T05:29:42.853255+0000","last_became_peered":"2023-11-04T05:29:42.853255+0000","last_unstale":"2023-11-04T05:29:44.044354+0000","last_undegraded":"2023-11-04T05:29:44.044354+0000","last_fullsized":"2023-11-04T05:29:44.044354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:49:16.749870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856346+0000","last_change":"2023-11-04T05:29:42.852973+0000","last_active":"2023-11-04T05:29:42.856346+0000","last_peered":"2023-11-04T05:29:42.856346+0000","last_clean":"2023-11-04T05:29:42.856346+0000","last_became_active":"2023-11-04T05:29:42.852680+0000","last_became_peered":"2023-11-04T05:29:42.852680+0000","last_unstale":"2023-11-04T05:29:42.856346+0000","last_undegraded":"2023-11-04T05:29:42.856346+0000","last_fullsized":"2023-11-04T05:29:42.856346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:32:03.063701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043919+0000","last_change":"2023-11-04T05:29:42.851462+0000","last_active":"2023-11-04T05:29:44.043919+0000","last_peered":"2023-11-04T05:29:44.043919+0000","last_clean":"2023-11-04T05:29:44.043919+0000","last_became_active":"2023-11-04T05:29:42.850955+0000","last_became_peered":"2023-11-04T05:29:42.850955+0000","last_unstale":"2023-11-04T05:29:44.043919+0000","last_undegraded":"2023-11-04T05:29:44.043919+0000","last_fullsized":"2023-11-04T05:29:44.043919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:33:47.033100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839709+0000","last_change":"2023-11-04T05:29:42.849177+0000","last_active":"2023-11-04T05:29:43.839709+0000","last_peered":"2023-11-04T05:29:43.839709+0000","last_clean":"2023-11-04T05:29:43.839709+0000","last_became_active":"2023-11-04T05:29:42.848905+0000","last_became_peered":"2023-11-04T05:29:42.848905+0000","last_unstale":"2023-11-04T05:29:43.839709+0000","last_undegraded":"2023-11-04T05:29:43.839709+0000","last_fullsized":"2023-11-04T05:29:43.839709+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:39:30.390449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043784+0000","last_change":"2023-11-04T05:29:42.855140+0000","last_active":"2023-11-04T05:29:44.043784+0000","last_peered":"2023-11-04T05:29:44.043784+0000","last_clean":"2023-11-04T05:29:44.043784+0000","last_became_active":"2023-11-04T05:29:42.854854+0000","last_became_peered":"2023-11-04T05:29:42.854854+0000","last_unstale":"2023-11-04T05:29:44.043784+0000","last_undegraded":"2023-11-04T05:29:44.043784+0000","last_fullsized":"2023-11-04T05:29:44.043784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:52:31.106845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838737+0000","last_change":"2023-11-04T05:29:42.847281+0000","last_active":"2023-11-04T05:29:43.838737+0000","last_peered":"2023-11-04T05:29:43.838737+0000","last_clean":"2023-11-04T05:29:43.838737+0000","last_became_active":"2023-11-04T05:29:42.846333+0000","last_became_peered":"2023-11-04T05:29:42.846333+0000","last_unstale":"2023-11-04T05:29:43.838737+0000","last_undegraded":"2023-11-04T05:29:43.838737+0000","last_fullsized":"2023-11-04T05:29:43.838737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:43:13.164856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839470+0000","last_change":"2023-11-04T05:29:42.856256+0000","last_active":"2023-11-04T05:29:43.839470+0000","last_peered":"2023-11-04T05:29:43.839470+0000","last_clean":"2023-11-04T05:29:43.839470+0000","last_became_active":"2023-11-04T05:29:42.855935+0000","last_became_peered":"2023-11-04T05:29:42.855935+0000","last_unstale":"2023-11-04T05:29:43.839470+0000","last_undegraded":"2023-11-04T05:29:43.839470+0000","last_fullsized":"2023-11-04T05:29:43.839470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:47:36.247887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843568+0000","last_change":"2023-11-04T05:29:42.847691+0000","last_active":"2023-11-04T05:29:43.843568+0000","last_peered":"2023-11-04T05:29:43.843568+0000","last_clean":"2023-11-04T05:29:43.843568+0000","last_became_active":"2023-11-04T05:29:42.847352+0000","last_became_peered":"2023-11-04T05:29:42.847352+0000","last_unstale":"2023-11-04T05:29:43.843568+0000","last_undegraded":"2023-11-04T05:29:43.843568+0000","last_fullsized":"2023-11-04T05:29:43.843568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:40:17.203021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856584+0000","last_change":"2023-11-04T05:29:42.851801+0000","last_active":"2023-11-04T05:29:42.856584+0000","last_peered":"2023-11-04T05:29:42.856584+0000","last_clean":"2023-11-04T05:29:42.856584+0000","last_became_active":"2023-11-04T05:29:42.851547+0000","last_became_peered":"2023-11-04T05:29:42.851547+0000","last_unstale":"2023-11-04T05:29:42.856584+0000","last_undegraded":"2023-11-04T05:29:42.856584+0000","last_fullsized":"2023-11-04T05:29:42.856584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:39:56.746730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843418+0000","last_change":"2023-11-04T05:29:42.852379+0000","last_active":"2023-11-04T05:29:43.843418+0000","last_peered":"2023-11-04T05:29:43.843418+0000","last_clean":"2023-11-04T05:29:43.843418+0000","last_became_active":"2023-11-04T05:29:42.852096+0000","last_became_peered":"2023-11-04T05:29:42.852096+0000","last_unstale":"2023-11-04T05:29:43.843418+0000","last_undegraded":"2023-11-04T05:29:43.843418+0000","last_fullsized":"2023-11-04T05:29:43.843418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:34:33.323379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043953+0000","last_change":"2023-11-04T05:29:42.851264+0000","last_active":"2023-11-04T05:29:44.043953+0000","last_peered":"2023-11-04T05:29:44.043953+0000","last_clean":"2023-11-04T05:29:44.043953+0000","last_became_active":"2023-11-04T05:29:42.850608+0000","last_became_peered":"2023-11-04T05:29:42.850608+0000","last_unstale":"2023-11-04T05:29:44.043953+0000","last_undegraded":"2023-11-04T05:29:44.043953+0000","last_fullsized":"2023-11-04T05:29:44.043953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:31:35.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839884+0000","last_change":"2023-11-04T05:29:42.855896+0000","last_active":"2023-11-04T05:29:43.839884+0000","last_peered":"2023-11-04T05:29:43.839884+0000","last_clean":"2023-11-04T05:29:43.839884+0000","last_became_active":"2023-11-04T05:29:42.855545+0000","last_became_peered":"2023-11-04T05:29:42.855545+0000","last_unstale":"2023-11-04T05:29:43.839884+0000","last_undegraded":"2023-11-04T05:29:43.839884+0000","last_fullsized":"2023-11-04T05:29:43.839884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:22:29.361864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839994+0000","last_change":"2023-11-04T05:29:42.856316+0000","last_active":"2023-11-04T05:29:43.839994+0000","last_peered":"2023-11-04T05:29:43.839994+0000","last_clean":"2023-11-04T05:29:43.839994+0000","last_became_active":"2023-11-04T05:29:42.855696+0000","last_became_peered":"2023-11-04T05:29:42.855696+0000","last_unstale":"2023-11-04T05:29:43.839994+0000","last_undegraded":"2023-11-04T05:29:43.839994+0000","last_fullsized":"2023-11-04T05:29:43.839994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:31:35.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856732+0000","last_change":"2023-11-04T05:29:42.852388+0000","last_active":"2023-11-04T05:29:42.856732+0000","last_peered":"2023-11-04T05:29:42.856732+0000","last_clean":"2023-11-04T05:29:42.856732+0000","last_became_active":"2023-11-04T05:29:42.852078+0000","last_became_peered":"2023-11-04T05:29:42.852078+0000","last_unstale":"2023-11-04T05:29:42.856732+0000","last_undegraded":"2023-11-04T05:29:42.856732+0000","last_fullsized":"2023-11-04T05:29:42.856732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:23:19.733640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843618+0000","last_change":"2023-11-04T05:29:42.849330+0000","last_active":"2023-11-04T05:29:43.843618+0000","last_peered":"2023-11-04T05:29:43.843618+0000","last_clean":"2023-11-04T05:29:43.843618+0000","last_became_active":"2023-11-04T05:29:42.849017+0000","last_became_peered":"2023-11-04T05:29:42.849017+0000","last_unstale":"2023-11-04T05:29:43.843618+0000","last_undegraded":"2023-11-04T05:29:43.843618+0000","last_fullsized":"2023-11-04T05:29:43.843618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:28:10.802723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.860604+0000","last_change":"2023-11-04T05:29:42.856713+0000","last_active":"2023-11-04T05:29:42.860604+0000","last_peered":"2023-11-04T05:29:42.860604+0000","last_clean":"2023-11-04T05:29:42.860604+0000","last_became_active":"2023-11-04T05:29:42.856441+0000","last_became_peered":"2023-11-04T05:29:42.856441+0000","last_unstale":"2023-11-04T05:29:42.860604+0000","last_undegraded":"2023-11-04T05:29:42.860604+0000","last_fullsized":"2023-11-04T05:29:42.860604+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:28:10.802723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843608+0000","last_change":"2023-11-04T05:29:42.853519+0000","last_active":"2023-11-04T05:29:43.843608+0000","last_peered":"2023-11-04T05:29:43.843608+0000","last_clean":"2023-11-04T05:29:43.843608+0000","last_became_active":"2023-11-04T05:29:42.853233+0000","last_became_peered":"2023-11-04T05:29:42.853233+0000","last_unstale":"2023-11-04T05:29:43.843608+0000","last_undegraded":"2023-11-04T05:29:43.843608+0000","last_fullsized":"2023-11-04T05:29:43.843608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:30:24.898958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044136+0000","last_change":"2023-11-04T05:29:42.851273+0000","last_active":"2023-11-04T05:29:44.044136+0000","last_peered":"2023-11-04T05:29:44.044136+0000","last_clean":"2023-11-04T05:29:44.044136+0000","last_became_active":"2023-11-04T05:29:42.850985+0000","last_became_peered":"2023-11-04T05:29:42.850985+0000","last_unstale":"2023-11-04T05:29:44.044136+0000","last_undegraded":"2023-11-04T05:29:44.044136+0000","last_fullsized":"2023-11-04T05:29:44.044136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:57:23.290722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843386+0000","last_change":"2023-11-04T05:29:42.851236+0000","last_active":"2023-11-04T05:29:43.843386+0000","last_peered":"2023-11-04T05:29:43.843386+0000","last_clean":"2023-11-04T05:29:43.843386+0000","last_became_active":"2023-11-04T05:29:42.850943+0000","last_became_peered":"2023-11-04T05:29:42.850943+0000","last_unstale":"2023-11-04T05:29:43.843386+0000","last_undegraded":"2023-11-04T05:29:43.843386+0000","last_fullsized":"2023-11-04T05:29:43.843386+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:32.226284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843352+0000","last_change":"2023-11-04T05:29:42.846479+0000","last_active":"2023-11-04T05:29:43.843352+0000","last_peered":"2023-11-04T05:29:43.843352+0000","last_clean":"2023-11-04T05:29:43.843352+0000","last_became_active":"2023-11-04T05:29:42.846163+0000","last_became_peered":"2023-11-04T05:29:42.846163+0000","last_unstale":"2023-11-04T05:29:43.843352+0000","last_undegraded":"2023-11-04T05:29:43.843352+0000","last_fullsized":"2023-11-04T05:29:43.843352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:55:46.446134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839847+0000","last_change":"2023-11-04T05:29:42.855935+0000","last_active":"2023-11-04T05:29:43.839847+0000","last_peered":"2023-11-04T05:29:43.839847+0000","last_clean":"2023-11-04T05:29:43.839847+0000","last_became_active":"2023-11-04T05:29:42.853857+0000","last_became_peered":"2023-11-04T05:29:42.853857+0000","last_unstale":"2023-11-04T05:29:43.839847+0000","last_undegraded":"2023-11-04T05:29:43.839847+0000","last_fullsized":"2023-11-04T05:29:43.839847+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:38:32.412729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.857986+0000","last_change":"2023-11-04T05:29:42.855533+0000","last_active":"2023-11-04T05:29:42.857986+0000","last_peered":"2023-11-04T05:29:42.857986+0000","last_clean":"2023-11-04T05:29:42.857986+0000","last_became_active":"2023-11-04T05:29:42.855214+0000","last_became_peered":"2023-11-04T05:29:42.855214+0000","last_unstale":"2023-11-04T05:29:42.857986+0000","last_undegraded":"2023-11-04T05:29:42.857986+0000","last_fullsized":"2023-11-04T05:29:42.857986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:13:09.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839321+0000","last_change":"2023-11-04T05:29:42.856342+0000","last_active":"2023-11-04T05:29:43.839321+0000","last_peered":"2023-11-04T05:29:43.839321+0000","last_clean":"2023-11-04T05:29:43.839321+0000","last_became_active":"2023-11-04T05:29:42.856024+0000","last_became_peered":"2023-11-04T05:29:42.856024+0000","last_unstale":"2023-11-04T05:29:43.839321+0000","last_undegraded":"2023-11-04T05:29:43.839321+0000","last_fullsized":"2023-11-04T05:29:43.839321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:48:46.815700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843673+0000","last_change":"2023-11-04T05:29:42.244294+0000","last_active":"2023-11-04T05:29:43.843673+0000","last_peered":"2023-11-04T05:29:43.843673+0000","last_clean":"2023-11-04T05:29:43.843673+0000","last_became_active":"2023-11-04T05:29:42.243967+0000","last_became_peered":"2023-11-04T05:29:42.243967+0000","last_unstale":"2023-11-04T05:29:43.843673+0000","last_undegraded":"2023-11-04T05:29:43.843673+0000","last_fullsized":"2023-11-04T05:29:43.843673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:18:22.886731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.857952+0000","last_change":"2023-11-04T05:29:42.855431+0000","last_active":"2023-11-04T05:29:42.857952+0000","last_peered":"2023-11-04T05:29:42.857952+0000","last_clean":"2023-11-04T05:29:42.857952+0000","last_became_active":"2023-11-04T05:29:42.855062+0000","last_became_peered":"2023-11-04T05:29:42.855062+0000","last_unstale":"2023-11-04T05:29:42.857952+0000","last_undegraded":"2023-11-04T05:29:42.857952+0000","last_fullsized":"2023-11-04T05:29:42.857952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:49:44.251317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.242671+0000","last_change":"2023-11-04T05:29:42.250993+0000","last_active":"2023-11-04T05:29:43.242671+0000","last_peered":"2023-11-04T05:29:43.242671+0000","last_clean":"2023-11-04T05:29:43.242671+0000","last_became_active":"2023-11-04T05:29:42.249979+0000","last_became_peered":"2023-11-04T05:29:42.249979+0000","last_unstale":"2023-11-04T05:29:43.242671+0000","last_undegraded":"2023-11-04T05:29:43.242671+0000","last_fullsized":"2023-11-04T05:29:43.242671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:39:29.384933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839651+0000","last_change":"2023-11-04T05:29:42.845095+0000","last_active":"2023-11-04T05:29:43.839651+0000","last_peered":"2023-11-04T05:29:43.839651+0000","last_clean":"2023-11-04T05:29:43.839651+0000","last_became_active":"2023-11-04T05:29:42.844812+0000","last_became_peered":"2023-11-04T05:29:42.844812+0000","last_unstale":"2023-11-04T05:29:43.839651+0000","last_undegraded":"2023-11-04T05:29:43.839651+0000","last_fullsized":"2023-11-04T05:29:43.839651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:33:43.907236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.247051+0000","last_change":"2023-11-04T05:29:42.250623+0000","last_active":"2023-11-04T05:29:43.247051+0000","last_peered":"2023-11-04T05:29:43.247051+0000","last_clean":"2023-11-04T05:29:43.247051+0000","last_became_active":"2023-11-04T05:29:42.250258+0000","last_became_peered":"2023-11-04T05:29:42.250258+0000","last_unstale":"2023-11-04T05:29:43.247051+0000","last_undegraded":"2023-11-04T05:29:43.247051+0000","last_fullsized":"2023-11-04T05:29:43.247051+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:55:45.440618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856157+0000","last_change":"2023-11-04T05:29:42.853117+0000","last_active":"2023-11-04T05:29:42.856157+0000","last_peered":"2023-11-04T05:29:42.856157+0000","last_clean":"2023-11-04T05:29:42.856157+0000","last_became_active":"2023-11-04T05:29:42.852846+0000","last_became_peered":"2023-11-04T05:29:42.852846+0000","last_unstale":"2023-11-04T05:29:42.856157+0000","last_undegraded":"2023-11-04T05:29:42.856157+0000","last_fullsized":"2023-11-04T05:29:42.856157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:48:46.815700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.241762+0000","last_change":"2023-11-04T05:29:42.247540+0000","last_active":"2023-11-04T05:29:43.241762+0000","last_peered":"2023-11-04T05:29:43.241762+0000","last_clean":"2023-11-04T05:29:43.241762+0000","last_became_active":"2023-11-04T05:29:42.245639+0000","last_became_peered":"2023-11-04T05:29:42.245639+0000","last_unstale":"2023-11-04T05:29:43.241762+0000","last_undegraded":"2023-11-04T05:29:43.241762+0000","last_fullsized":"2023-11-04T05:29:43.241762+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:32:02.058185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.852133+0000","last_change":"2023-11-04T05:29:42.848581+0000","last_active":"2023-11-04T05:29:42.852133+0000","last_peered":"2023-11-04T05:29:42.852133+0000","last_clean":"2023-11-04T05:29:42.852133+0000","last_became_active":"2023-11-04T05:29:42.847752+0000","last_became_peered":"2023-11-04T05:29:42.847752+0000","last_unstale":"2023-11-04T05:29:42.852133+0000","last_undegraded":"2023-11-04T05:29:42.852133+0000","last_fullsized":"2023-11-04T05:29:42.852133+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:28:11.515210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044025+0000","last_change":"2023-11-04T05:29:42.256925+0000","last_active":"2023-11-04T05:29:44.044025+0000","last_peered":"2023-11-04T05:29:44.044025+0000","last_clean":"2023-11-04T05:29:44.044025+0000","last_became_active":"2023-11-04T05:29:42.256678+0000","last_became_peered":"2023-11-04T05:29:42.256678+0000","last_unstale":"2023-11-04T05:29:44.044025+0000","last_undegraded":"2023-11-04T05:29:44.044025+0000","last_fullsized":"2023-11-04T05:29:44.044025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:43:12.159340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043791+0000","last_change":"2023-11-04T05:29:42.855173+0000","last_active":"2023-11-04T05:29:44.043791+0000","last_peered":"2023-11-04T05:29:44.043791+0000","last_clean":"2023-11-04T05:29:44.043791+0000","last_became_active":"2023-11-04T05:29:42.854885+0000","last_became_peered":"2023-11-04T05:29:42.854885+0000","last_unstale":"2023-11-04T05:29:44.043791+0000","last_undegraded":"2023-11-04T05:29:44.043791+0000","last_fullsized":"2023-11-04T05:29:44.043791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:38:32.412729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838950+0000","last_change":"2023-11-04T05:29:42.251387+0000","last_active":"2023-11-04T05:29:43.838950+0000","last_peered":"2023-11-04T05:29:43.838950+0000","last_clean":"2023-11-04T05:29:43.838950+0000","last_became_active":"2023-11-04T05:29:42.251014+0000","last_became_peered":"2023-11-04T05:29:42.251014+0000","last_unstale":"2023-11-04T05:29:43.838950+0000","last_undegraded":"2023-11-04T05:29:43.838950+0000","last_fullsized":"2023-11-04T05:29:43.838950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:31.220768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839777+0000","last_change":"2023-11-04T05:29:42.855754+0000","last_active":"2023-11-04T05:29:43.839777+0000","last_peered":"2023-11-04T05:29:43.839777+0000","last_clean":"2023-11-04T05:29:43.839777+0000","last_became_active":"2023-11-04T05:29:42.855188+0000","last_became_peered":"2023-11-04T05:29:42.855188+0000","last_unstale":"2023-11-04T05:29:43.839777+0000","last_undegraded":"2023-11-04T05:29:43.839777+0000","last_fullsized":"2023-11-04T05:29:43.839777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:57:18.580597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.841805+0000","last_change":"2023-11-04T05:29:42.250268+0000","last_active":"2023-11-04T05:29:42.841805+0000","last_peered":"2023-11-04T05:29:42.841805+0000","last_clean":"2023-11-04T05:29:42.841805+0000","last_became_active":"2023-11-04T05:29:42.249994+0000","last_became_peered":"2023-11-04T05:29:42.249994+0000","last_unstale":"2023-11-04T05:29:42.841805+0000","last_undegraded":"2023-11-04T05:29:42.841805+0000","last_fullsized":"2023-11-04T05:29:42.841805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:34:32.317863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851449+0000","last_change":"2023-11-04T05:29:42.848295+0000","last_active":"2023-11-04T05:29:42.851449+0000","last_peered":"2023-11-04T05:29:42.851449+0000","last_clean":"2023-11-04T05:29:42.851449+0000","last_became_active":"2023-11-04T05:29:42.848039+0000","last_became_peered":"2023-11-04T05:29:42.848039+0000","last_unstale":"2023-11-04T05:29:42.851449+0000","last_undegraded":"2023-11-04T05:29:42.851449+0000","last_fullsized":"2023-11-04T05:29:42.851449+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:43:13.164856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.836503+0000","last_change":"2023-11-04T05:29:42.243435+0000","last_active":"2023-11-04T05:29:42.836503+0000","last_peered":"2023-11-04T05:29:42.836503+0000","last_clean":"2023-11-04T05:29:42.836503+0000","last_became_active":"2023-11-04T05:29:42.243164+0000","last_became_peered":"2023-11-04T05:29:42.243164+0000","last_unstale":"2023-11-04T05:29:42.836503+0000","last_undegraded":"2023-11-04T05:29:42.836503+0000","last_fullsized":"2023-11-04T05:29:42.836503+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:09:48.727495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856515+0000","last_change":"2023-11-04T05:29:42.851240+0000","last_active":"2023-11-04T05:29:42.856515+0000","last_peered":"2023-11-04T05:29:42.856515+0000","last_clean":"2023-11-04T05:29:42.856515+0000","last_became_active":"2023-11-04T05:29:42.850969+0000","last_became_peered":"2023-11-04T05:29:42.850969+0000","last_unstale":"2023-11-04T05:29:42.856515+0000","last_undegraded":"2023-11-04T05:29:42.856515+0000","last_fullsized":"2023-11-04T05:29:42.856515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:32:07.906761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.844672+0000","last_change":"2023-11-04T05:29:42.254224+0000","last_active":"2023-11-04T05:29:42.844672+0000","last_peered":"2023-11-04T05:29:42.844672+0000","last_clean":"2023-11-04T05:29:42.844672+0000","last_became_active":"2023-11-04T05:29:42.253917+0000","last_became_peered":"2023-11-04T05:29:42.253917+0000","last_unstale":"2023-11-04T05:29:42.844672+0000","last_undegraded":"2023-11-04T05:29:42.844672+0000","last_fullsized":"2023-11-04T05:29:42.844672+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:31:34.371973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839513+0000","last_change":"2023-11-04T05:29:42.851132+0000","last_active":"2023-11-04T05:29:43.839513+0000","last_peered":"2023-11-04T05:29:43.839513+0000","last_clean":"2023-11-04T05:29:43.839513+0000","last_became_active":"2023-11-04T05:29:42.850862+0000","last_became_peered":"2023-11-04T05:29:42.850862+0000","last_unstale":"2023-11-04T05:29:43.839513+0000","last_undegraded":"2023-11-04T05:29:43.839513+0000","last_fullsized":"2023-11-04T05:29:43.839513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:52:31.106845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044286+0000","last_change":"2023-11-04T05:29:42.245665+0000","last_active":"2023-11-04T05:29:44.044286+0000","last_peered":"2023-11-04T05:29:44.044286+0000","last_clean":"2023-11-04T05:29:44.044286+0000","last_became_active":"2023-11-04T05:29:42.245390+0000","last_became_peered":"2023-11-04T05:29:42.245390+0000","last_unstale":"2023-11-04T05:29:44.044286+0000","last_undegraded":"2023-11-04T05:29:44.044286+0000","last_fullsized":"2023-11-04T05:29:44.044286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:48:45.810184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.855647+0000","last_change":"2023-11-04T05:29:42.851283+0000","last_active":"2023-11-04T05:29:42.855647+0000","last_peered":"2023-11-04T05:29:42.855647+0000","last_clean":"2023-11-04T05:29:42.855647+0000","last_became_active":"2023-11-04T05:29:42.851033+0000","last_became_peered":"2023-11-04T05:29:42.851033+0000","last_unstale":"2023-11-04T05:29:42.855647+0000","last_undegraded":"2023-11-04T05:29:42.855647+0000","last_fullsized":"2023-11-04T05:29:42.855647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:20:03.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044421+0000","last_change":"2023-11-04T05:29:42.251479+0000","last_active":"2023-11-04T05:29:44.044421+0000","last_peered":"2023-11-04T05:29:44.044421+0000","last_clean":"2023-11-04T05:29:44.044421+0000","last_became_active":"2023-11-04T05:29:42.251178+0000","last_became_peered":"2023-11-04T05:29:42.251178+0000","last_unstale":"2023-11-04T05:29:44.044421+0000","last_undegraded":"2023-11-04T05:29:44.044421+0000","last_fullsized":"2023-11-04T05:29:44.044421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:03:56.966857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839287+0000","last_change":"2023-11-04T05:29:42.854496+0000","last_active":"2023-11-04T05:29:43.839287+0000","last_peered":"2023-11-04T05:29:43.839287+0000","last_clean":"2023-11-04T05:29:43.839287+0000","last_became_active":"2023-11-04T05:29:42.854159+0000","last_became_peered":"2023-11-04T05:29:42.854159+0000","last_unstale":"2023-11-04T05:29:43.839287+0000","last_undegraded":"2023-11-04T05:29:43.839287+0000","last_fullsized":"2023-11-04T05:29:43.839287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:44:41.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.245046+0000","last_change":"2023-11-04T05:29:42.246203+0000","last_active":"2023-11-04T05:29:43.245046+0000","last_peered":"2023-11-04T05:29:43.245046+0000","last_clean":"2023-11-04T05:29:43.245046+0000","last_became_active":"2023-11-04T05:29:42.245738+0000","last_became_peered":"2023-11-04T05:29:42.245738+0000","last_unstale":"2023-11-04T05:29:43.245046+0000","last_undegraded":"2023-11-04T05:29:43.245046+0000","last_fullsized":"2023-11-04T05:29:43.245046+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:33:46.027584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.855172+0000","last_change":"2023-11-04T05:29:42.851355+0000","last_active":"2023-11-04T05:29:42.855172+0000","last_peered":"2023-11-04T05:29:42.855172+0000","last_clean":"2023-11-04T05:29:42.855172+0000","last_became_active":"2023-11-04T05:29:42.850687+0000","last_became_peered":"2023-11-04T05:29:42.850687+0000","last_unstale":"2023-11-04T05:29:42.855172+0000","last_undegraded":"2023-11-04T05:29:42.855172+0000","last_fullsized":"2023-11-04T05:29:42.855172+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:53.484326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.240802+0000","last_change":"2023-11-04T05:29:42.251484+0000","last_active":"2023-11-04T05:29:43.240802+0000","last_peered":"2023-11-04T05:29:43.240802+0000","last_clean":"2023-11-04T05:29:43.240802+0000","last_became_active":"2023-11-04T05:29:42.251195+0000","last_became_peered":"2023-11-04T05:29:42.251195+0000","last_unstale":"2023-11-04T05:29:43.240802+0000","last_undegraded":"2023-11-04T05:29:43.240802+0000","last_fullsized":"2023-11-04T05:29:43.240802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:13:13.966265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.853190+0000","last_change":"2023-11-04T05:29:42.851604+0000","last_active":"2023-11-04T05:29:42.853190+0000","last_peered":"2023-11-04T05:29:42.853190+0000","last_clean":"2023-11-04T05:29:42.853190+0000","last_became_active":"2023-11-04T05:29:42.851144+0000","last_became_peered":"2023-11-04T05:29:42.851144+0000","last_unstale":"2023-11-04T05:29:42.853190+0000","last_undegraded":"2023-11-04T05:29:42.853190+0000","last_fullsized":"2023-11-04T05:29:42.853190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:47:27.907828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.842000+0000","last_change":"2023-11-04T05:29:42.249779+0000","last_active":"2023-11-04T05:29:42.842000+0000","last_peered":"2023-11-04T05:29:42.842000+0000","last_clean":"2023-11-04T05:29:42.842000+0000","last_became_active":"2023-11-04T05:29:42.249471+0000","last_became_peered":"2023-11-04T05:29:42.249471+0000","last_unstale":"2023-11-04T05:29:42.842000+0000","last_undegraded":"2023-11-04T05:29:42.842000+0000","last_fullsized":"2023-11-04T05:29:42.842000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:40:16.197505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838872+0000","last_change":"2023-11-04T05:29:42.854025+0000","last_active":"2023-11-04T05:29:43.838872+0000","last_peered":"2023-11-04T05:29:43.838872+0000","last_clean":"2023-11-04T05:29:43.838872+0000","last_became_active":"2023-11-04T05:29:42.853738+0000","last_became_peered":"2023-11-04T05:29:42.853738+0000","last_unstale":"2023-11-04T05:29:43.838872+0000","last_undegraded":"2023-11-04T05:29:43.838872+0000","last_fullsized":"2023-11-04T05:29:43.838872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:48:51.090986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843228+0000","last_change":"2023-11-04T05:29:42.244946+0000","last_active":"2023-11-04T05:29:43.843228+0000","last_peered":"2023-11-04T05:29:43.843228+0000","last_clean":"2023-11-04T05:29:43.843228+0000","last_became_active":"2023-11-04T05:29:42.244637+0000","last_became_peered":"2023-11-04T05:29:42.244637+0000","last_unstale":"2023-11-04T05:29:43.843228+0000","last_undegraded":"2023-11-04T05:29:43.843228+0000","last_fullsized":"2023-11-04T05:29:43.843228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:37:59.024785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856132+0000","last_change":"2023-11-04T05:29:42.853943+0000","last_active":"2023-11-04T05:29:42.856132+0000","last_peered":"2023-11-04T05:29:42.856132+0000","last_clean":"2023-11-04T05:29:42.856132+0000","last_became_active":"2023-11-04T05:29:42.853686+0000","last_became_peered":"2023-11-04T05:29:42.853686+0000","last_unstale":"2023-11-04T05:29:42.856132+0000","last_undegraded":"2023-11-04T05:29:42.856132+0000","last_fullsized":"2023-11-04T05:29:42.856132+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:03:57.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.838835+0000","last_change":"2023-11-04T05:29:42.246692+0000","last_active":"2023-11-04T05:29:43.838835+0000","last_peered":"2023-11-04T05:29:43.838835+0000","last_clean":"2023-11-04T05:29:43.838835+0000","last_became_active":"2023-11-04T05:29:42.245957+0000","last_became_peered":"2023-11-04T05:29:42.245957+0000","last_unstale":"2023-11-04T05:29:43.838835+0000","last_undegraded":"2023-11-04T05:29:43.838835+0000","last_fullsized":"2023-11-04T05:29:43.838835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:52:48.500536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851058+0000","last_change":"2023-11-04T05:29:42.849176+0000","last_active":"2023-11-04T05:29:42.851058+0000","last_peered":"2023-11-04T05:29:42.851058+0000","last_clean":"2023-11-04T05:29:42.851058+0000","last_became_active":"2023-11-04T05:29:42.848947+0000","last_became_peered":"2023-11-04T05:29:42.848947+0000","last_unstale":"2023-11-04T05:29:42.851058+0000","last_undegraded":"2023-11-04T05:29:42.851058+0000","last_fullsized":"2023-11-04T05:29:42.851058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:51:25.063057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044383+0000","last_change":"2023-11-04T05:29:42.245183+0000","last_active":"2023-11-04T05:29:44.044383+0000","last_peered":"2023-11-04T05:29:44.044383+0000","last_clean":"2023-11-04T05:29:44.044383+0000","last_became_active":"2023-11-04T05:29:42.244897+0000","last_became_peered":"2023-11-04T05:29:42.244897+0000","last_unstale":"2023-11-04T05:29:44.044383+0000","last_undegraded":"2023-11-04T05:29:44.044383+0000","last_fullsized":"2023-11-04T05:29:44.044383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:48:50.085470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044356+0000","last_change":"2023-11-04T05:29:42.853518+0000","last_active":"2023-11-04T05:29:44.044356+0000","last_peered":"2023-11-04T05:29:44.044356+0000","last_clean":"2023-11-04T05:29:44.044356+0000","last_became_active":"2023-11-04T05:29:42.852829+0000","last_became_peered":"2023-11-04T05:29:42.852829+0000","last_unstale":"2023-11-04T05:29:44.044356+0000","last_undegraded":"2023-11-04T05:29:44.044356+0000","last_fullsized":"2023-11-04T05:29:44.044356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:11.307467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.242865+0000","last_change":"2023-11-04T05:29:42.244225+0000","last_active":"2023-11-04T05:29:43.242865+0000","last_peered":"2023-11-04T05:29:43.242865+0000","last_clean":"2023-11-04T05:29:43.242865+0000","last_became_active":"2023-11-04T05:29:42.243961+0000","last_became_peered":"2023-11-04T05:29:42.243961+0000","last_unstale":"2023-11-04T05:29:43.242865+0000","last_undegraded":"2023-11-04T05:29:43.242865+0000","last_fullsized":"2023-11-04T05:29:43.242865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T14:57:52.478810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839930+0000","last_change":"2023-11-04T05:29:42.849081+0000","last_active":"2023-11-04T05:29:43.839930+0000","last_peered":"2023-11-04T05:29:43.839930+0000","last_clean":"2023-11-04T05:29:43.839930+0000","last_became_active":"2023-11-04T05:29:42.848634+0000","last_became_peered":"2023-11-04T05:29:42.848634+0000","last_unstale":"2023-11-04T05:29:43.839930+0000","last_undegraded":"2023-11-04T05:29:43.839930+0000","last_fullsized":"2023-11-04T05:29:43.839930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T13:47:27.348365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.241713+0000","last_change":"2023-11-04T05:29:42.249437+0000","last_active":"2023-11-04T05:29:43.241713+0000","last_peered":"2023-11-04T05:29:43.241713+0000","last_clean":"2023-11-04T05:29:43.241713+0000","last_became_active":"2023-11-04T05:29:42.249113+0000","last_became_peered":"2023-11-04T05:29:42.249113+0000","last_unstale":"2023-11-04T05:29:43.241713+0000","last_undegraded":"2023-11-04T05:29:43.241713+0000","last_fullsized":"2023-11-04T05:29:43.241713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:07:14.642689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856667+0000","last_change":"2023-11-04T05:29:42.852274+0000","last_active":"2023-11-04T05:29:42.856667+0000","last_peered":"2023-11-04T05:29:42.856667+0000","last_clean":"2023-11-04T05:29:42.856667+0000","last_became_active":"2023-11-04T05:29:42.852008+0000","last_became_peered":"2023-11-04T05:29:42.852008+0000","last_unstale":"2023-11-04T05:29:42.856667+0000","last_undegraded":"2023-11-04T05:29:42.856667+0000","last_fullsized":"2023-11-04T05:29:42.856667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:35:37.907113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.842367+0000","last_change":"2023-11-04T05:29:42.254132+0000","last_active":"2023-11-04T05:29:42.842367+0000","last_peered":"2023-11-04T05:29:42.842367+0000","last_clean":"2023-11-04T05:29:42.842367+0000","last_became_active":"2023-11-04T05:29:42.253781+0000","last_became_peered":"2023-11-04T05:29:42.253781+0000","last_unstale":"2023-11-04T05:29:42.842367+0000","last_undegraded":"2023-11-04T05:29:42.842367+0000","last_fullsized":"2023-11-04T05:29:42.842367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:30:23.893442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839470+0000","last_change":"2023-11-04T05:29:42.853230+0000","last_active":"2023-11-04T05:29:43.839470+0000","last_peered":"2023-11-04T05:29:43.839470+0000","last_clean":"2023-11-04T05:29:43.839470+0000","last_became_active":"2023-11-04T05:29:42.852935+0000","last_became_peered":"2023-11-04T05:29:42.852935+0000","last_unstale":"2023-11-04T05:29:43.839470+0000","last_undegraded":"2023-11-04T05:29:43.839470+0000","last_fullsized":"2023-11-04T05:29:43.839470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:27.494898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.240720+0000","last_change":"2023-11-04T05:29:42.256298+0000","last_active":"2023-11-04T05:29:43.240720+0000","last_peered":"2023-11-04T05:29:43.240720+0000","last_clean":"2023-11-04T05:29:43.240720+0000","last_became_active":"2023-11-04T05:29:42.255985+0000","last_became_peered":"2023-11-04T05:29:42.255985+0000","last_unstale":"2023-11-04T05:29:43.240720+0000","last_undegraded":"2023-11-04T05:29:43.240720+0000","last_fullsized":"2023-11-04T05:29:43.240720+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:49:23.763058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.855250+0000","last_change":"2023-11-04T05:29:42.850893+0000","last_active":"2023-11-04T05:29:42.855250+0000","last_peered":"2023-11-04T05:29:42.855250+0000","last_clean":"2023-11-04T05:29:42.855250+0000","last_became_active":"2023-11-04T05:29:42.850594+0000","last_became_peered":"2023-11-04T05:29:42.850594+0000","last_unstale":"2023-11-04T05:29:42.855250+0000","last_undegraded":"2023-11-04T05:29:42.855250+0000","last_fullsized":"2023-11-04T05:29:42.855250+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T07:33:43.907236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.843194+0000","last_change":"2023-11-04T05:29:42.246052+0000","last_active":"2023-11-04T05:29:42.843194+0000","last_peered":"2023-11-04T05:29:42.843194+0000","last_clean":"2023-11-04T05:29:42.843194+0000","last_became_active":"2023-11-04T05:29:42.245710+0000","last_became_peered":"2023-11-04T05:29:42.245710+0000","last_unstale":"2023-11-04T05:29:42.843194+0000","last_undegraded":"2023-11-04T05:29:42.843194+0000","last_fullsized":"2023-11-04T05:29:42.843194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:31:10.799253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.043928+0000","last_change":"2023-11-04T05:29:42.851945+0000","last_active":"2023-11-04T05:29:44.043928+0000","last_peered":"2023-11-04T05:29:44.043928+0000","last_clean":"2023-11-04T05:29:44.043928+0000","last_became_active":"2023-11-04T05:29:42.851668+0000","last_became_peered":"2023-11-04T05:29:42.851668+0000","last_unstale":"2023-11-04T05:29:44.043928+0000","last_undegraded":"2023-11-04T05:29:44.043928+0000","last_fullsized":"2023-11-04T05:29:44.043928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:13:14.971781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.840052+0000","last_change":"2023-11-04T05:29:42.249736+0000","last_active":"2023-11-04T05:29:43.840052+0000","last_peered":"2023-11-04T05:29:43.840052+0000","last_clean":"2023-11-04T05:29:43.840052+0000","last_became_active":"2023-11-04T05:29:42.249345+0000","last_became_peered":"2023-11-04T05:29:42.249345+0000","last_unstale":"2023-11-04T05:29:43.840052+0000","last_undegraded":"2023-11-04T05:29:43.840052+0000","last_fullsized":"2023-11-04T05:29:43.840052+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:47:26.902313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851468+0000","last_change":"2023-11-04T05:29:42.848886+0000","last_active":"2023-11-04T05:29:42.851468+0000","last_peered":"2023-11-04T05:29:42.851468+0000","last_clean":"2023-11-04T05:29:42.851468+0000","last_became_active":"2023-11-04T05:29:42.848579+0000","last_became_peered":"2023-11-04T05:29:42.848579+0000","last_unstale":"2023-11-04T05:29:42.851468+0000","last_undegraded":"2023-11-04T05:29:42.851468+0000","last_fullsized":"2023-11-04T05:29:42.851468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:48:51.090986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.841401+0000","last_change":"2023-11-04T05:29:42.250049+0000","last_active":"2023-11-04T05:29:42.841401+0000","last_peered":"2023-11-04T05:29:42.841401+0000","last_clean":"2023-11-04T05:29:42.841401+0000","last_became_active":"2023-11-04T05:29:42.249781+0000","last_became_peered":"2023-11-04T05:29:42.249781+0000","last_unstale":"2023-11-04T05:29:42.841401+0000","last_undegraded":"2023-11-04T05:29:42.841401+0000","last_fullsized":"2023-11-04T05:29:42.841401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T16:54:56.672880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.858003+0000","last_change":"2023-11-04T05:29:42.856102+0000","last_active":"2023-11-04T05:29:42.858003+0000","last_peered":"2023-11-04T05:29:42.858003+0000","last_clean":"2023-11-04T05:29:42.858003+0000","last_became_active":"2023-11-04T05:29:42.855832+0000","last_became_peered":"2023-11-04T05:29:42.855832+0000","last_unstale":"2023-11-04T05:29:42.858003+0000","last_undegraded":"2023-11-04T05:29:42.858003+0000","last_fullsized":"2023-11-04T05:29:42.858003+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T06:35:50.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044444+0000","last_change":"2023-11-04T05:29:42.249712+0000","last_active":"2023-11-04T05:29:44.044444+0000","last_peered":"2023-11-04T05:29:44.044444+0000","last_clean":"2023-11-04T05:29:44.044444+0000","last_became_active":"2023-11-04T05:29:42.249435+0000","last_became_peered":"2023-11-04T05:29:42.249435+0000","last_unstale":"2023-11-04T05:29:44.044444+0000","last_undegraded":"2023-11-04T05:29:44.044444+0000","last_fullsized":"2023-11-04T05:29:44.044444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:26.489383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839858+0000","last_change":"2023-11-04T05:29:42.853782+0000","last_active":"2023-11-04T05:29:43.839858+0000","last_peered":"2023-11-04T05:29:43.839858+0000","last_clean":"2023-11-04T05:29:43.839858+0000","last_became_active":"2023-11-04T05:29:42.853476+0000","last_became_peered":"2023-11-04T05:29:42.853476+0000","last_unstale":"2023-11-04T05:29:43.839858+0000","last_undegraded":"2023-11-04T05:29:43.839858+0000","last_fullsized":"2023-11-04T05:29:43.839858+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T17:13:14.971781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.841609+0000","last_change":"2023-11-04T05:29:42.244493+0000","last_active":"2023-11-04T05:29:42.841609+0000","last_peered":"2023-11-04T05:29:42.841609+0000","last_clean":"2023-11-04T05:29:42.841609+0000","last_became_active":"2023-11-04T05:29:42.244176+0000","last_became_peered":"2023-11-04T05:29:42.244176+0000","last_unstale":"2023-11-04T05:29:42.841609+0000","last_undegraded":"2023-11-04T05:29:42.841609+0000","last_fullsized":"2023-11-04T05:29:42.841609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:39:40.090868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.851054+0000","last_change":"2023-11-04T05:29:42.847667+0000","last_active":"2023-11-04T05:29:42.851054+0000","last_peered":"2023-11-04T05:29:42.851054+0000","last_clean":"2023-11-04T05:29:42.851054+0000","last_became_active":"2023-11-04T05:29:42.847385+0000","last_became_peered":"2023-11-04T05:29:42.847385+0000","last_unstale":"2023-11-04T05:29:42.851054+0000","last_undegraded":"2023-11-04T05:29:42.851054+0000","last_fullsized":"2023-11-04T05:29:42.851054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:07:27.494898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.839730+0000","last_change":"2023-11-04T05:29:42.251723+0000","last_active":"2023-11-04T05:29:42.839730+0000","last_peered":"2023-11-04T05:29:42.839730+0000","last_clean":"2023-11-04T05:29:42.839730+0000","last_became_active":"2023-11-04T05:29:42.251469+0000","last_became_peered":"2023-11-04T05:29:42.251469+0000","last_unstale":"2023-11-04T05:29:42.839730+0000","last_undegraded":"2023-11-04T05:29:42.839730+0000","last_fullsized":"2023-11-04T05:29:42.839730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T11:49:38.372915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.852111+0000","last_change":"2023-11-04T05:29:42.845934+0000","last_active":"2023-11-04T05:29:42.852111+0000","last_peered":"2023-11-04T05:29:42.852111+0000","last_clean":"2023-11-04T05:29:42.852111+0000","last_became_active":"2023-11-04T05:29:42.845654+0000","last_became_peered":"2023-11-04T05:29:42.845654+0000","last_unstale":"2023-11-04T05:29:42.852111+0000","last_undegraded":"2023-11-04T05:29:42.852111+0000","last_fullsized":"2023-11-04T05:29:42.852111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:56:20.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.840104+0000","last_change":"2023-11-04T05:29:42.253414+0000","last_active":"2023-11-04T05:29:42.840104+0000","last_peered":"2023-11-04T05:29:42.840104+0000","last_clean":"2023-11-04T05:29:42.840104+0000","last_became_active":"2023-11-04T05:29:42.253131+0000","last_became_peered":"2023-11-04T05:29:42.253131+0000","last_unstale":"2023-11-04T05:29:42.840104+0000","last_undegraded":"2023-11-04T05:29:42.840104+0000","last_fullsized":"2023-11-04T05:29:42.840104+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:18:16.548604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.856392+0000","last_change":"2023-11-04T05:29:42.853402+0000","last_active":"2023-11-04T05:29:42.856392+0000","last_peered":"2023-11-04T05:29:42.856392+0000","last_clean":"2023-11-04T05:29:42.856392+0000","last_became_active":"2023-11-04T05:29:42.853145+0000","last_became_peered":"2023-11-04T05:29:42.853145+0000","last_unstale":"2023-11-04T05:29:42.856392+0000","last_undegraded":"2023-11-04T05:29:42.856392+0000","last_fullsized":"2023-11-04T05:29:42.856392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:18:23.892247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.843308+0000","last_change":"2023-11-04T05:29:42.246860+0000","last_active":"2023-11-04T05:29:43.843308+0000","last_peered":"2023-11-04T05:29:43.843308+0000","last_clean":"2023-11-04T05:29:43.843308+0000","last_became_active":"2023-11-04T05:29:42.246588+0000","last_became_peered":"2023-11-04T05:29:42.246588+0000","last_unstale":"2023-11-04T05:29:43.843308+0000","last_undegraded":"2023-11-04T05:29:43.843308+0000","last_fullsized":"2023-11-04T05:29:43.843308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T05:49:43.245801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839676+0000","last_change":"2023-11-04T05:29:42.846158+0000","last_active":"2023-11-04T05:29:43.839676+0000","last_peered":"2023-11-04T05:29:43.839676+0000","last_clean":"2023-11-04T05:29:43.839676+0000","last_became_active":"2023-11-04T05:29:42.845876+0000","last_became_peered":"2023-11-04T05:29:42.845876+0000","last_unstale":"2023-11-04T05:29:43.839676+0000","last_undegraded":"2023-11-04T05:29:43.839676+0000","last_fullsized":"2023-11-04T05:29:43.839676+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:20:03.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.836545+0000","last_change":"2023-11-04T05:29:42.247803+0000","last_active":"2023-11-04T05:29:42.836545+0000","last_peered":"2023-11-04T05:29:42.836545+0000","last_clean":"2023-11-04T05:29:42.836545+0000","last_became_active":"2023-11-04T05:29:42.245715+0000","last_became_peered":"2023-11-04T05:29:42.245715+0000","last_unstale":"2023-11-04T05:29:42.836545+0000","last_undegraded":"2023-11-04T05:29:42.836545+0000","last_fullsized":"2023-11-04T05:29:42.836545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T15:03:21.108953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839209+0000","last_change":"2023-11-04T05:29:42.845180+0000","last_active":"2023-11-04T05:29:43.839209+0000","last_peered":"2023-11-04T05:29:43.839209+0000","last_clean":"2023-11-04T05:29:43.839209+0000","last_became_active":"2023-11-04T05:29:42.844842+0000","last_became_peered":"2023-11-04T05:29:42.844842+0000","last_unstale":"2023-11-04T05:29:43.839209+0000","last_undegraded":"2023-11-04T05:29:43.839209+0000","last_fullsized":"2023-11-04T05:29:43.839209+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:03:57.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.843565+0000","last_change":"2023-11-04T05:29:42.248779+0000","last_active":"2023-11-04T05:29:42.843565+0000","last_peered":"2023-11-04T05:29:42.843565+0000","last_clean":"2023-11-04T05:29:42.843565+0000","last_became_active":"2023-11-04T05:29:42.248476+0000","last_became_peered":"2023-11-04T05:29:42.248476+0000","last_unstale":"2023-11-04T05:29:42.843565+0000","last_undegraded":"2023-11-04T05:29:42.843565+0000","last_fullsized":"2023-11-04T05:29:42.843565+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T08:52:30.101329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:44.044306+0000","last_change":"2023-11-04T05:29:42.852568+0000","last_active":"2023-11-04T05:29:44.044306+0000","last_peered":"2023-11-04T05:29:44.044306+0000","last_clean":"2023-11-04T05:29:44.044306+0000","last_became_active":"2023-11-04T05:29:42.852246+0000","last_became_peered":"2023-11-04T05:29:42.852246+0000","last_unstale":"2023-11-04T05:29:44.044306+0000","last_undegraded":"2023-11-04T05:29:44.044306+0000","last_fullsized":"2023-11-04T05:29:44.044306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:42.222954+0000","last_clean_scrub_stamp":"2023-11-04T05:29:42.222954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T12:58:39.441935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-04T05:29:43.839399+0000","last_change":"2023-11-04T05:29:42.251343+0000","last_active":"2023-11-04T05:29:43.839399+0000","last_peered":"2023-11-04T05:29:43.839399+0000","last_clean":"2023-11-04T05:29:43.839399+0000","last_became_active":"2023-11-04T05:29:42.251040+0000","last_became_peered":"2023-11-04T05:29:42.251040+0000","last_unstale":"2023-11-04T05:29:43.839399+0000","last_undegraded":"2023-11-04T05:29:43.839399+0000","last_fullsized":"2023-11-04T05:29:43.839399+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:29:41.217438+0000","last_clean_scrub_stamp":"2023-11-04T05:29:41.217438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T09:30:23.893442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-04T05:29:42.837201+0000","last_change":"2023-11-04T05:28:11.829985+0000","last_active":"2023-11-04T05:29:42.837201+0000","last_peered":"2023-11-04T05:29:42.837201+0000","last_clean":"2023-11-04T05:29:42.837201+0000","last_became_active":"2023-11-04T05:28:11.829705+0000","last_became_peered":"2023-11-04T05:28:11.829705+0000","last_unstale":"2023-11-04T05:29:42.837201+0000","last_undegraded":"2023-11-04T05:29:42.837201+0000","last_fullsized":"2023-11-04T05:29:42.837201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-04T05:28:10.378947+0000","last_clean_scrub_stamp":"2023-11-04T05:28:10.378947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-05T10:46:15.100605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1064960,"data_stored":787652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.8480000000000008,"5min":8.8480000000000008,"15min":8.8480000000000008},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.6020000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.7300000000000004,"5min":8.7300000000000004,"15min":8.7300000000000004},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.9410000000000007,"5min":8.9410000000000007,"15min":8.9410000000000007},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.7549999999999999,"5min":7.7549999999999999,"15min":7.7549999999999999},"last":0.73299999999999998}]},{"osd":3,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.0909999999999993,"5min":9.0909999999999993,"15min":9.0909999999999993},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.70399999999999996}]},{"osd":4,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.89900000000000002}]},{"osd":6,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71999999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":458752,"data_stored":196184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34499999999999997}]},{"osd":3,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.891,"5min":3.891,"15min":3.891},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.59299999999999997}]},{"osd":7,"last update":"Sat Nov 4 05:29:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.60999999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":779350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":11.079000000000001,"5min":11.079000000000001,"15min":11.079000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":10.798999999999999,"5min":10.798999999999999,"15min":10.798999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":11.958,"5min":11.958,"15min":11.958},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.100000000000001,"5min":20.100000000000001,"15min":20.100000000000001},"last":0.45700000000000002}]},{"osd":3,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":11.021000000000001,"5min":11.021000000000001,"15min":11.021000000000001},"last":0.433},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":15.153,"5min":15.153,"15min":15.153},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":10.939,"5min":10.939,"15min":10.939},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.99,"5min":10.99,"15min":10.99},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":10.879,"5min":10.879,"15min":10.879},"last":0.749},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":10.981999999999999,"5min":10.981999999999999,"15min":10.981999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":11.048,"5min":11.048,"15min":11.048},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":10.840999999999999,"5min":10.840999999999999,"15min":10.840999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Sat Nov 4 05:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":10.920999999999999,"5min":10.920999999999999,"15min":10.920999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":11.018000000000001,"5min":11.018000000000001,"15min":11.018000000000001},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":462848,"data_stored":196206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":3.988,"5min":3.988,"15min":3.988},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":5.694,"5min":5.694,"15min":5.694},"last":0.44400000000000001}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":9.7789999999999999,"5min":9.7789999999999999,"15min":9.7789999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.7729999999999997,"5min":5.7729999999999997,"15min":5.7729999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.8740000000000001,"5min":3.8740000000000001,"15min":3.8740000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.927,"5min":3.927,"15min":3.927},"last":0.61399999999999999}]},{"osd":5,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.505}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60499999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":189072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.63}]},{"osd":4,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.755}]},{"osd":6,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Sat Nov 4 05:29:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81200000000000006}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.38800000000000001}]},{"osd":3,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63600000000000001}]},{"osd":5,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.624},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66200000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":190628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.51400000000000001}]},{"osd":2,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64000000000000001}]},{"osd":4,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.77200000000000002}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.87},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75600000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":189094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":8.0540000000000003,"5min":8.0540000000000003,"15min":8.0540000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":10.067,"5min":10.067,"15min":10.067},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":8.1300000000000008,"5min":8.1300000000000008,"15min":8.1300000000000008},"last":0.36699999999999999}]},{"osd":2,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":9.0890000000000004,"5min":9.0890000000000004,"15min":9.0890000000000004},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":10.169,"5min":10.169,"15min":10.169},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005}]},{"osd":7,"last update":"Sat Nov 4 05:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-04T05:29:47.805 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-04T05:29:47.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-04T05:29:47.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.969+0000 7fdff77a4700 1 Processor -- start 2023-11-04T05:29:47.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.970+0000 7fdff77a4700 1 -- start start 2023-11-04T05:29:47.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.970+0000 7fdff77a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff0072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.970+0000 7fdff77a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdff015ba20 0x7fdff00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.970+0000 7fdff77a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff015b590 con 0x7fdff00cf3e0 2023-11-04T05:29:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.970+0000 7fdff77a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff0073230 con 0x7fdff015ba20 2023-11-04T05:29:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.970+0000 7fdff77a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff015ad30 con 0x7fdff00b5d50 2023-11-04T05:29:47.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff5d41700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdff00cf3e0 0x7fdff00cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59448/0 (socket says 172.21.15.3:59448) 2023-11-04T05:29:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff5d41700 1 -- 172.21.15.3:0/3309005628 learned_addr learned my addr 172.21.15.3:0/3309005628 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff4d3f700 1 --2- 172.21.15.3:0/3309005628 >> v2:172.21.15.3:3300/0 conn(0x7fdff015ba20 0x7fdff00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdfe7fff700 1 -- 172.21.15.3:0/3309005628 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455112961 0 0) 0x7fdff015b590 con 0x7fdff00cf3e0 2023-11-04T05:29:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff5540700 1 --2- 172.21.15.3:0/3309005628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff0072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdfe7fff700 1 -- 172.21.15.3:0/3309005628 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfdc0035b0 con 0x7fdff00cf3e0 2023-11-04T05:29:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff4d3f700 1 -- 172.21.15.3:0/3309005628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 msgr2=0x7fdff0072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff4d3f700 1 --2- 172.21.15.3:0/3309005628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff0072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff4d3f700 1 -- 172.21.15.3:0/3309005628 >> v1:172.21.15.3:6789/0 conn(0x7fdff00cf3e0 legacy=0x7fdff00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff4d3f700 1 -- 172.21.15.3:0/3309005628 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00d01c0 con 0x7fdff015ba20 2023-11-04T05:29:47.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdff4d3f700 1 --2- 172.21.15.3:0/3309005628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00b2a80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdff015ae70 tx=0x7fdfec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bc4004c5308047 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdfe7fff700 1 -- 172.21.15.3:0/3309005628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004020 con 0x7fdff015ba20 2023-11-04T05:29:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdfe7fff700 1 -- 172.21.15.3:0/3309005628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec004180 con 0x7fdff015ba20 2023-11-04T05:29:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.971+0000 7fdfe7fff700 1 -- 172.21.15.3:0/3309005628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004530 con 0x7fdff015ba20 2023-11-04T05:29:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 -- 172.21.15.3:0/3309005628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 msgr2=0x7fdff00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3309005628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00b2a80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdff015ae70 tx=0x7fdfec009b20 comp rx=0 tx=0).stop 2023-11-04T05:29:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 -- 172.21.15.3:0/3309005628 shutdown_connections 2023-11-04T05:29:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3309005628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff0072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3309005628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00b2a80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 -- 172.21.15.3:0/3309005628 >> 172.21.15.3:0/3309005628 conn(0x7fdff006def0 msgr2=0x7fdff006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:47.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 -- 172.21.15.3:0/3309005628 shutdown_connections 2023-11-04T05:29:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.972+0000 7fdff77a4700 1 -- 172.21.15.3:0/3309005628 wait complete. 2023-11-04T05:29:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.973+0000 7fdff77a4700 1 Processor -- start 2023-11-04T05:29:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.973+0000 7fdff77a4700 1 -- start start 2023-11-04T05:29:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff77a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff01499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff77a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff00cf3e0 0x7fdff00c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff77a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff77a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff01f3900 con 0x7fdff015ba20 2023-11-04T05:29:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff77a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff01f3b50 con 0x7fdff00cf3e0 2023-11-04T05:29:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff77a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff01f3e30 con 0x7fdff00b5d50 2023-11-04T05:29:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54106/0 (socket says 172.21.15.3:54106) 2023-11-04T05:29:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 -- 172.21.15.3:0/3056518417 learned_addr learned my addr 172.21.15.3:0/3056518417 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5540700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff01499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff4d3f700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff00cf3e0 0x7fdff00c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 -- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 msgr2=0x7fdff01499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff01499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 -- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff00cf3e0 msgr2=0x7fdff00c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff00cf3e0 0x7fdff00c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5d41700 1 -- 172.21.15.3:0/3056518417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0094a0 con 0x7fdff015ba20 2023-11-04T05:29:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff5540700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff01499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:47.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.974+0000 7fdff4d3f700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff00cf3e0 0x7fdff00c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.975+0000 7fdff5d41700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00c5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800cb40 tx=0x7fdfe800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.975+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800ab50 con 0x7fdff015ba20 2023-11-04T05:29:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.975+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800acb0 con 0x7fdff015ba20 2023-11-04T05:29:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.975+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff01f4110 con 0x7fdff015ba20 2023-11-04T05:29:47.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.975+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8012620 con 0x7fdff015ba20 2023-11-04T05:29:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.975+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff01f46d0 con 0x7fdff015ba20 2023-11-04T05:29:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.976+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0074df0 con 0x7fdff015ba20 2023-11-04T05:29:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.976+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfe801b3b0 con 0x7fdff015ba20 2023-11-04T05:29:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.977+0000 7fdfe5ffb700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdfd406af00 0x7fdfd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.977+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdfe8089510 con 0x7fdff015ba20 2023-11-04T05:29:47.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.977+0000 7fdff5540700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdfd406af00 0x7fdfd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:47.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.978+0000 7fdff5540700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdfd406af00 0x7fdfd406d3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009820 tx=0x7fdfe0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:47.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:47.981+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fdfe8054480 con 0x7fdff015ba20 2023-11-04T05:29:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.266+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdff0071fc0 con 0x7fdff015ba20 2023-11-04T05:29:48.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.268+0000 7fdfe5ffb700 1 -- 172.21.15.3:0/3056518417 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdfe8057aa0 con 0x7fdff015ba20 2023-11-04T05:29:48.270 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-11-04T05:29:48.270 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA81kVl1BIWGBAAag9+v589VkQNzEcgULo1sw== 2023-11-04T05:29:48.270 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2023-11-04T05:29:48.270 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2023-11-04T05:29:48.271 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-04T05:29:48.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdfd406af00 msgr2=0x7fdfd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdfd406af00 0x7fdfd406d3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009820 tx=0x7fdfe0008040 comp rx=0 tx=0).stop 2023-11-04T05:29:48.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 msgr2=0x7fdff00c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00c5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800cb40 tx=0x7fdfe800cf00 comp rx=0 tx=0).stop 2023-11-04T05:29:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 shutdown_connections 2023-11-04T05:29:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdfd406af00 0x7fdfd406d3c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdff015ba20 0x7fdff00c5700 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdff00b5d50 0x7fdff01499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 --2- 172.21.15.3:0/3056518417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff00cf3e0 0x7fdff00c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 >> 172.21.15.3:0/3056518417 conn(0x7fdff006def0 msgr2=0x7fdff00ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 shutdown_connections 2023-11-04T05:29:48.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.270+0000 7fdff77a4700 1 -- 172.21.15.3:0/3056518417 wait complete. 2023-11-04T05:29:48.275 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-11-04T05:29:48.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-04T05:29:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.408+0000 7ff311db3700 1 Processor -- start 2023-11-04T05:29:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.408+0000 7ff311db3700 1 -- start start 2023-11-04T05:29:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff311db3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c1612c0 0x7ff30c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff311db3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c0b2e00 0x7ff30c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:48.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff311db3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c1616a0 con 0x7ff30c0d1f80 2023-11-04T05:29:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff311db3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c0b2bb0 con 0x7ff30c1612c0 2023-11-04T05:29:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff311db3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0d42c0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff30b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c1612c0 0x7ff30c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:48.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff30b7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c1612c0 0x7ff30c06bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54110/0 (socket says 172.21.15.3:54110) 2023-11-04T05:29:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.409+0000 7ff30b7fe700 1 -- 172.21.15.3:0/674330070 learned_addr learned my addr 172.21.15.3:0/674330070 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30affd700 1 --2- 172.21.15.3:0/674330070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c0b2e00 0x7ff30c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30b7fe700 1 -- 172.21.15.3:0/674330070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c0b2e00 msgr2=0x7ff30c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/674330070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c0b2e00 0x7ff30c0d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30b7fe700 1 -- 172.21.15.3:0/674330070 >> v1:172.21.15.188:6789/0 conn(0x7ff30c0d1f80 legacy=0x7ff30c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30b7fe700 1 -- 172.21.15.3:0/674330070 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c0d45a0 con 0x7ff30c1612c0 2023-11-04T05:29:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/674330070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c1612c0 0x7ff30c06bd30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006fd0 tx=0x7ff2fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a2b2558c4b0f26 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30a7fc700 1 -- 172.21.15.3:0/674330070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc005e30 con 0x7ff30c1612c0 2023-11-04T05:29:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30a7fc700 1 -- 172.21.15.3:0/674330070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc004030 con 0x7ff30c1612c0 2023-11-04T05:29:48.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.410+0000 7ff30a7fc700 1 -- 172.21.15.3:0/674330070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0043e0 con 0x7ff30c1612c0 2023-11-04T05:29:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 -- 172.21.15.3:0/674330070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c1612c0 msgr2=0x7ff30c06bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 --2- 172.21.15.3:0/674330070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c1612c0 0x7ff30c06bd30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006fd0 tx=0x7ff2fc009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 -- 172.21.15.3:0/674330070 shutdown_connections 2023-11-04T05:29:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 --2- 172.21.15.3:0/674330070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c1612c0 0x7ff30c06bd30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 --2- 172.21.15.3:0/674330070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c0b2e00 0x7ff30c0d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 -- 172.21.15.3:0/674330070 >> 172.21.15.3:0/674330070 conn(0x7ff30c08aff0 msgr2=0x7ff30c0d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 -- 172.21.15.3:0/674330070 shutdown_connections 2023-11-04T05:29:48.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.411+0000 7ff311db3700 1 -- 172.21.15.3:0/674330070 wait complete. 2023-11-04T05:29:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.412+0000 7ff311db3700 1 Processor -- start 2023-11-04T05:29:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.413+0000 7ff311db3700 1 -- start start 2023-11-04T05:29:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 0x7ff30c1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff30c0d1f80 0x7ff30c0b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:48.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c1612c0 0x7ff30c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1f4c90 con 0x7ff30c0b2e00 2023-11-04T05:29:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1f4940 con 0x7ff30c0d1f80 2023-11-04T05:29:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0b7940 con 0x7ff30c1612c0 2023-11-04T05:29:48.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 0x7ff30c1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 0x7ff30c1f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54116/0 (socket says 172.21.15.3:54116) 2023-11-04T05:29:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 -- 172.21.15.3:0/2115872955 learned_addr learned my addr 172.21.15.3:0/2115872955 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30affd700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff30c0d1f80 0x7ff30c0b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30bfff700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c1612c0 0x7ff30c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 -- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c1612c0 msgr2=0x7ff30c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c1612c0 0x7ff30c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 -- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff30c0d1f80 msgr2=0x7ff30c0b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff30c0d1f80 0x7ff30c0b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 -- 172.21.15.3:0/2115872955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc009500 con 0x7ff30c0b2e00 2023-11-04T05:29:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30affd700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff30c0d1f80 0x7ff30c0b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:48.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30bfff700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c1612c0 0x7ff30c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:48.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff30b7fe700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 0x7ff30c1f6fe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009130 tx=0x7ff2fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:48.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc003d90 con 0x7ff30c0b2e00 2023-11-04T05:29:48.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc011420 con 0x7ff30c0b2e00 2023-11-04T05:29:48.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0117d0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c0b7c20 con 0x7ff30c0b2e00 2023-11-04T05:29:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.414+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c1f7520 con 0x7ff30c0b2e00 2023-11-04T05:29:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.415+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c1f4dd0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.416+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2fc011930 con 0x7ff30c0b2e00 2023-11-04T05:29:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.416+0000 7ff303fff700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2ec06af00 0x7ff2ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:48.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.416+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff2fc08a9d0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.416+0000 7ff30affd700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2ec06af00 0x7ff2ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.417+0000 7ff30affd700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2ec06af00 0x7ff2ec06d3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff30c1f6cd0 tx=0x7ff2f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:48.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.421+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff30c1f4dd0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.668+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30c1f4dd0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.669+0000 7ff303fff700 1 -- 172.21.15.3:0/2115872955 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4901 (secure 0 0 0) 0x7ff30c1f4dd0 con 0x7ff30c0b2e00 2023-11-04T05:29:48.671 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:48.672 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2614810755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2614810755},{"type":"v1","addr":"172.21.15.188:6836","nonce":2614810755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5023,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/37625371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":37625371},{"type":"v1","addr":"172.21.15.188:6837","nonce":37625371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1141511123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1141511123},{"type":"v1","addr":"172.21.15.3:6834","nonce":1141511123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-04T05:29:42.650628+0000","modified":"2023-11-04T05:29:45.668525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/64074701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":64074701},{"type":"v1","addr":"172.21.15.3:6835","nonce":64074701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-04T05:29:48.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2ec06af00 msgr2=0x7ff2ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2ec06af00 0x7ff2ec06d3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff30c1f6cd0 tx=0x7ff2f4008040 comp rx=0 tx=0).stop 2023-11-04T05:29:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 msgr2=0x7ff30c1f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 0x7ff30c1f6fe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009130 tx=0x7ff2fc004750 comp rx=0 tx=0).stop 2023-11-04T05:29:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 shutdown_connections 2023-11-04T05:29:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff2ec06af00 0x7ff2ec06d3c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff30c1612c0 0x7ff30c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0b2e00 0x7ff30c1f6fe0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 --2- 172.21.15.3:0/2115872955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff30c0d1f80 0x7ff30c0b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 >> 172.21.15.3:0/2115872955 conn(0x7ff30c08aff0 msgr2=0x7ff30c0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:48.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 shutdown_connections 2023-11-04T05:29:48.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:48.671+0000 7ff311db3700 1 -- 172.21.15.3:0/2115872955 wait complete. 2023-11-04T05:29:48.676 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2023-11-04T05:29:49.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-04T05:29:42.650628+0000', 'modified': '2023-11-04T05:29:45.668525+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5379': {'gid': 5379, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/64074701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6833', 'nonce': 64074701}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 64074701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-04T05:29:49.089 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-04T05:29:49.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-04T05:29:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.217+0000 7f837965f700 1 Processor -- start 2023-11-04T05:29:49.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.217+0000 7f837965f700 1 -- start start 2023-11-04T05:29:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f837965f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f837415ddd0 0x7f83740d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f837965f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83740d0310 0x7f83740d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f837965f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f83740d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f837965f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740cfca0 con 0x7f83740d0310 2023-11-04T05:29:49.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f837965f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740d0cb0 con 0x7f837415ddd0 2023-11-04T05:29:49.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f837965f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837406bef0 con 0x7f83740d1770 2023-11-04T05:29:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f83727fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83740d0310 0x7f83740d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f83727fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83740d0310 0x7f83740d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54124/0 (socket says 172.21.15.3:54124) 2023-11-04T05:29:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f83727fc700 1 -- 172.21.15.3:0/3199080431 learned_addr learned my addr 172.21.15.3:0/3199080431 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.218+0000 7f8372ffd700 1 --2- 172.21.15.3:0/3199080431 >> v2:172.21.15.188:3300/0 conn(0x7f837415ddd0 0x7f83740d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83737fe700 1 --2- 172.21.15.3:0/3199080431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f83740d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83727fc700 1 -- 172.21.15.3:0/3199080431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 msgr2=0x7f83740d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83727fc700 1 --2- 172.21.15.3:0/3199080431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f83740d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83727fc700 1 -- 172.21.15.3:0/3199080431 >> v2:172.21.15.188:3300/0 conn(0x7f837415ddd0 msgr2=0x7f83740d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83727fc700 1 --2- 172.21.15.3:0/3199080431 >> v2:172.21.15.188:3300/0 conn(0x7f837415ddd0 0x7f83740d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83727fc700 1 -- 172.21.15.3:0/3199080431 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740cf400 con 0x7f83740d0310 2023-11-04T05:29:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f8372ffd700 1 --2- 172.21.15.3:0/3199080431 >> v2:172.21.15.188:3300/0 conn(0x7f837415ddd0 0x7f83740d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f83727fc700 1 --2- 172.21.15.3:0/3199080431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740d0770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f837408c740 tx=0x7f8364009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff8bbe9060f3072 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f8371ffb700 1 -- 172.21.15.3:0/3199080431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400b870 con 0x7f83740d0310 2023-11-04T05:29:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f8371ffb700 1 -- 172.21.15.3:0/3199080431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836400b9d0 con 0x7f83740d0310 2023-11-04T05:29:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.219+0000 7f8371ffb700 1 -- 172.21.15.3:0/3199080431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400bd60 con 0x7f83740d0310 2023-11-04T05:29:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 -- 172.21.15.3:0/3199080431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 msgr2=0x7f83740d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 --2- 172.21.15.3:0/3199080431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740d0770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f837408c740 tx=0x7f8364009bd0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 -- 172.21.15.3:0/3199080431 shutdown_connections 2023-11-04T05:29:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 --2- 172.21.15.3:0/3199080431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f83740d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 --2- 172.21.15.3:0/3199080431 >> v2:172.21.15.188:3300/0 conn(0x7f837415ddd0 0x7f83740d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 --2- 172.21.15.3:0/3199080431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740d0770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 -- 172.21.15.3:0/3199080431 >> 172.21.15.3:0/3199080431 conn(0x7f83740d3900 msgr2=0x7f83740d3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 -- 172.21.15.3:0/3199080431 shutdown_connections 2023-11-04T05:29:49.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.220+0000 7f837965f700 1 -- 172.21.15.3:0/3199080431 wait complete. 2023-11-04T05:29:49.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.221+0000 7f837965f700 1 Processor -- start 2023-11-04T05:29:49.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.221+0000 7f837965f700 1 -- start start 2023-11-04T05:29:49.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f837965f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f837965f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f837414dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f837965f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f837415ddd0 0x7f83740d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f837965f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b7340 con 0x7f83740d0310 2023-11-04T05:29:49.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f837965f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837415e3c0 con 0x7f837415ddd0 2023-11-04T05:29:49.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f837965f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b8ff0 con 0x7f83740d1770 2023-11-04T05:29:49.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54128/0 (socket says 172.21.15.3:54128) 2023-11-04T05:29:49.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 -- 172.21.15.3:0/3803536017 learned_addr learned my addr 172.21.15.3:0/3803536017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f83737fe700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f837415ddd0 0x7f83740d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f83727fc700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f837414dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 -- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 msgr2=0x7f837414dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f837414dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 -- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f837415ddd0 msgr2=0x7f83740d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f837415ddd0 0x7f83740d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 -- 172.21.15.3:0/3803536017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364009550 con 0x7f83740d0310 2023-11-04T05:29:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f83727fc700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f837414dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f8372ffd700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740b8710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e840 tx=0x7f835c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.222+0000 7f836b7fe700 1 -- 172.21.15.3:0/3803536017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c004230 con 0x7f83740d0310 2023-11-04T05:29:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.223+0000 7f836b7fe700 1 -- 172.21.15.3:0/3803536017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c004390 con 0x7f83740d0310 2023-11-04T05:29:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.223+0000 7f836b7fe700 1 -- 172.21.15.3:0/3803536017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0036a0 con 0x7f83740d0310 2023-11-04T05:29:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.223+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83740b92d0 con 0x7f83740d0310 2023-11-04T05:29:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.223+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83740b8c50 con 0x7f83740d0310 2023-11-04T05:29:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.224+0000 7f836b7fe700 1 -- 172.21.15.3:0/3803536017 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f835c003800 con 0x7f83740d0310 2023-11-04T05:29:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.224+0000 7f836b7fe700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f835406af00 0x7f835406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.224+0000 7f836b7fe700 1 -- 172.21.15.3:0/3803536017 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f835c089190 con 0x7f83740d0310 2023-11-04T05:29:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.225+0000 7f83727fc700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f835406af00 0x7f835406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.225+0000 7f83727fc700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f835406af00 0x7f835406d3c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f83740b6360 tx=0x7f8364008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.226+0000 7f8385859b80 1 Processor -- start 2023-11-04T05:29:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 -- start start 2023-11-04T05:29:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e80c30 0x562677eae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7ead0 0x562677e98f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677dea6e0 con 0x562677e7e6f0 2023-11-04T05:29:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677dde3b0 con 0x562677e80c30 2023-11-04T05:29:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562677ddbcd0 con 0x562677e7ead0 2023-11-04T05:29:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8372ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8372ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54136/0 (socket says 172.21.15.3:54136) 2023-11-04T05:29:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.227+0000 7f8372ffd700 1 -- 172.21.15.3:0/2628039316 learned_addr learned my addr 172.21.15.3:0/2628039316 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f83727fc700 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e80c30 0x562677eae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f83737fe700 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7ead0 0x562677e98f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8372ffd700 1 -- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7ead0 msgr2=0x562677e98f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8372ffd700 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7ead0 0x562677e98f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8372ffd700 1 -- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e80c30 msgr2=0x562677eae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8372ffd700 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e80c30 0x562677eae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8372ffd700 1 -- 172.21.15.3:0/2628039316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562677c545e0 con 0x562677e7e6f0 2023-11-04T05:29:49.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8372ffd700 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eae450 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f835c05fb90 tx=0x7f835c058540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da21c37a54cda126 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8368ff9700 1 -- 172.21.15.3:0/2628039316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c057dc0 con 0x562677e7e6f0 2023-11-04T05:29:49.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8368ff9700 1 -- 172.21.15.3:0/2628039316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c05bf10 con 0x562677e7e6f0 2023-11-04T05:29:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.228+0000 7f8368ff9700 1 -- 172.21.15.3:0/2628039316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0600c0 con 0x562677e7e6f0 2023-11-04T05:29:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 -- 172.21.15.3:0/2628039316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 msgr2=0x562677eae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eae450 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f835c05fb90 tx=0x7f835c058540 comp rx=0 tx=0).stop 2023-11-04T05:29:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 -- 172.21.15.3:0/2628039316 shutdown_connections 2023-11-04T05:29:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7ead0 0x562677e98f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eae450 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 --2- 172.21.15.3:0/2628039316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e80c30 0x562677eae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 -- 172.21.15.3:0/2628039316 >> 172.21.15.3:0/2628039316 conn(0x562677df9660 msgr2=0x562677e843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 -- 172.21.15.3:0/2628039316 shutdown_connections 2023-11-04T05:29:49.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 -- 172.21.15.3:0/2628039316 wait complete. 2023-11-04T05:29:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.229+0000 7f8385859b80 1 -- start start 2023-11-04T05:29:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 Processor -- start 2023-11-04T05:29:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eb2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7ead0 0x562677ec1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677ec3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677dd75c0 con 0x562677e7e6f0 2023-11-04T05:29:49.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677d730d0 con 0x562677e7ead0 2023-11-04T05:29:49.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562677d72d70 con 0x562677ec23d0 2023-11-04T05:29:49.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8385859b80 10 client.0 _ll_register_callbacks cb 0x562677bcf7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-04T05:29:49.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.230+0000 7f8372ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eb2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eb2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54142/0 (socket says 172.21.15.3:54142) 2023-11-04T05:29:49.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 -- 172.21.15.3:0/4011959955 learned_addr learned my addr 172.21.15.3:0/4011959955 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f83727fc700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7ead0 0x562677ec1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f83737fe700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677ec3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 -- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 msgr2=0x562677ec3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677ec3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 -- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7ead0 msgr2=0x562677ec1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7ead0 0x562677ec1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 -- 172.21.15.3:0/4011959955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83740b8c50 con 0x562677e7e6f0 2023-11-04T05:29:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f83737fe700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677ec3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8372ffd700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eb2280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f835c05fb90 tx=0x7f835c00c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19b5fc6b8eef5d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-11-04T05:29:49.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8351ffb700 1 -- 172.21.15.3:0/4011959955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0600c0 con 0x562677e7e6f0 2023-11-04T05:29:49.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562677e76880 con 0x562677e7e6f0 2023-11-04T05:29:49.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8351ffb700 1 -- 172.21.15.3:0/4011959955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f835c05b070 con 0x562677e7e6f0 2023-11-04T05:29:49.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8351ffb700 1 -- 172.21.15.3:0/4011959955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f835c05fc30 con 0x562677e7e6f0 2023-11-04T05:29:49.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8385859b80 10 client.5550 fetch_fsmap learned FSMap version 18 2023-11-04T05:29:49.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x562677c00d90 con 0x562677e7e6f0 2023-11-04T05:29:49.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.231+0000 7f8351ffb700 1 -- 172.21.15.3:0/4011959955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562677e76880 con 0x562677e7e6f0 2023-11-04T05:29:49.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8351ffb700 1 -- 172.21.15.3:0/4011959955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f835c06bb10 con 0x562677e7e6f0 2023-11-04T05:29:49.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8385859b80 10 client.5550 fetch_fsmap finished waiting for FSMap version 18 2023-11-04T05:29:49.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8385859b80 10 client.5550 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-04T05:29:49.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8385859b80 20 client.5550 populate_metadata read hostname 'smithi003' 2023-11-04T05:29:49.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8385859b80 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677ed81e0 0x562677e8a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8385859b80 4 client.5550 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-04T05:29:49.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 --> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562677e52a30 con 0x562677ed81e0 2023-11-04T05:29:49.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f83727fc700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677ed81e0 0x562677e8a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f8351ffb700 10 client.5550 ms_handle_connect on v2:172.21.15.3:6833/64074701 2023-11-04T05:29:49.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.232+0000 7f83727fc700 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677ed81e0 0x562677e8a340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816bffe82d2e015c server_cookie=9f1db1a730807fad in_seq=0 out_seq=0 2023-11-04T05:29:49.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8351ffb700 1 -- 172.21.15.3:0/4011959955 <== mds.0 v2:172.21.15.3:6833/64074701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562677e52a30 con 0x562677ed81e0 2023-11-04T05:29:49.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8351ffb700 10 client.5550 handle_command_reply: tid=0 2023-11-04T05:29:49.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 client.5550 shutdown 2023-11-04T05:29:49.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 msgr2=0x562677eb2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eb2280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f835c05fb90 tx=0x7f835c00c1f0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 shutdown_connections 2023-11-04T05:29:49.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677ed81e0 0x562677e8a340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677ec3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7e6f0 0x562677eb2280 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 --2- 172.21.15.3:0/4011959955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7ead0 0x562677ec1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8351ffb700 0 client.5550 ms_handle_reset on v2:172.21.15.3:6833/64074701 2023-11-04T05:29:49.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 >> 172.21.15.3:0/4011959955 conn(0x562677df9660 msgr2=0x562677e7f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 shutdown_connections 2023-11-04T05:29:49.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 1 -- 172.21.15.3:0/4011959955 wait complete. 2023-11-04T05:29:49.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.233+0000 7f8385859b80 20 client.5550 trim_cache size 0 max 16384 2023-11-04T05:29:49.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.251+0000 7f8385859b80 1 Processor -- start 2023-11-04T05:29:49.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.251+0000 7f8385859b80 1 -- start start 2023-11-04T05:29:49.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7e6f0 0x562677e7f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8385859b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677ec0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677e86c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8385859b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677c568d0 con 0x562677e7ead0 2023-11-04T05:29:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677e822f0 con 0x562677e7e6f0 2023-11-04T05:29:49.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562677e82960 con 0x562677ec23d0 2023-11-04T05:29:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f83727fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677ec0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f83727fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677ec0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54152/0 (socket says 172.21.15.3:54152) 2023-11-04T05:29:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f83727fc700 1 -- 172.21.15.3:0/2977493453 learned_addr learned my addr 172.21.15.3:0/2977493453 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.252+0000 7f8372ffd700 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7e6f0 0x562677e7f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83737fe700 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677e86c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83727fc700 1 -- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 msgr2=0x562677e86c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83727fc700 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677e86c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83727fc700 1 -- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7e6f0 msgr2=0x562677e7f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83727fc700 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7e6f0 0x562677e7f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83727fc700 1 -- 172.21.15.3:0/2977493453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562677be9c10 con 0x562677e7ead0 2023-11-04T05:29:49.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83737fe700 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677e86c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:29:49.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83727fc700 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677ec0fe0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8364009c10 tx=0x7f8364011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97acd8694ab6f21f server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83697fa700 1 -- 172.21.15.3:0/2977493453 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836402c040 con 0x562677e7ead0 2023-11-04T05:29:49.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83697fa700 1 -- 172.21.15.3:0/2977493453 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364030530 con 0x562677e7ead0 2023-11-04T05:29:49.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f83697fa700 1 -- 172.21.15.3:0/2977493453 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364030a90 con 0x562677e7ead0 2023-11-04T05:29:49.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 -- 172.21.15.3:0/2977493453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 msgr2=0x562677ec0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677ec0fe0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8364009c10 tx=0x7f8364011000 comp rx=0 tx=0).stop 2023-11-04T05:29:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 -- 172.21.15.3:0/2977493453 shutdown_connections 2023-11-04T05:29:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677e7e6f0 0x562677e7f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677ec23d0 0x562677e86c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 --2- 172.21.15.3:0/2977493453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677ec0fe0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.253+0000 7f8385859b80 1 -- 172.21.15.3:0/2977493453 >> 172.21.15.3:0/2977493453 conn(0x562677df9660 msgr2=0x562677e7a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.254+0000 7f8385859b80 1 -- 172.21.15.3:0/2977493453 shutdown_connections 2023-11-04T05:29:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.254+0000 7f8385859b80 1 -- 172.21.15.3:0/2977493453 wait complete. 2023-11-04T05:29:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.254+0000 7f8385859b80 1 -- start start 2023-11-04T05:29:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.254+0000 7f8385859b80 1 Processor -- start 2023-11-04T05:29:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.255+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7e6f0 0x562677e7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.255+0000 7f8385859b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677e7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.255+0000 7f8385859b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677ec23d0 0x562677eda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.255+0000 7f8385859b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677e82e70 con 0x562677e7ead0 2023-11-04T05:29:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.255+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562677e7f860 con 0x562677ec23d0 2023-11-04T05:29:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.255+0000 7f8385859b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562677e82fe0 con 0x562677e7e6f0 2023-11-04T05:29:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8385859b80 10 client.0 _ll_register_callbacks cb 0x7f835c0575e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-04T05:29:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677e7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677e7da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54158/0 (socket says 172.21.15.3:54158) 2023-11-04T05:29:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 -- 172.21.15.3:0/1236968795 learned_addr learned my addr 172.21.15.3:0/1236968795 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8372ffd700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7e6f0 0x562677e7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83737fe700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677ec23d0 0x562677eda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 -- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7e6f0 msgr2=0x562677e7d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7e6f0 0x562677e7d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 -- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677ec23d0 msgr2=0x562677eda5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677ec23d0 0x562677eda5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 -- 172.21.15.3:0/1236968795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83740cf400 con 0x562677e7ead0 2023-11-04T05:29:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83737fe700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677ec23d0 0x562677eda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T05:29:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f83727fc700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677e7da40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f837408dca0 tx=0x7f83640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19d5d915e62cde server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-11-04T05:29:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8351ffb700 1 -- 172.21.15.3:0/1236968795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836402c070 con 0x562677e7ead0 2023-11-04T05:29:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8351ffb700 1 -- 172.21.15.3:0/1236968795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f83640036a0 con 0x562677e7ead0 2023-11-04T05:29:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f835c05bf10 con 0x562677e7ead0 2023-11-04T05:29:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8351ffb700 1 -- 172.21.15.3:0/1236968795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83640303b0 con 0x562677e7ead0 2023-11-04T05:29:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8385859b80 10 client.5556 fetch_fsmap learned FSMap version 18 2023-11-04T05:29:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.256+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x562677c27200 con 0x562677e7ead0 2023-11-04T05:29:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8351ffb700 1 -- 172.21.15.3:0/1236968795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c05bf10 con 0x562677e7ead0 2023-11-04T05:29:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8351ffb700 1 -- 172.21.15.3:0/1236968795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8364005a20 con 0x562677e7ead0 2023-11-04T05:29:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 10 client.5556 fetch_fsmap finished waiting for FSMap version 18 2023-11-04T05:29:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 10 client.5556 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-04T05:29:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 20 client.5556 populate_metadata read hostname 'smithi003' 2023-11-04T05:29:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677edae70 0x562677e8c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 4 client.5556 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-04T05:29:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 --> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562677dd05a0 con 0x562677edae70 2023-11-04T05:29:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8372ffd700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677edae70 0x562677e8c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8351ffb700 10 client.5556 ms_handle_connect on v2:172.21.15.3:6833/64074701 2023-11-04T05:29:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8372ffd700 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677edae70 0x562677e8c3b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5916eba77b3836e6 server_cookie=6d02168294210148 in_seq=0 out_seq=0 2023-11-04T05:29:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8351ffb700 1 -- 172.21.15.3:0/1236968795 <== mds.0 v2:172.21.15.3:6833/64074701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562677dd05a0 con 0x562677edae70 2023-11-04T05:29:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8351ffb700 10 client.5556 handle_command_reply: tid=0 2023-11-04T05:29:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 client.5556 shutdown 2023-11-04T05:29:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 msgr2=0x562677e7da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677e7da40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f837408dca0 tx=0x7f83640047c0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 shutdown_connections 2023-11-04T05:29:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:6833/64074701,v1:172.21.15.3:6835/64074701] conn(0x562677edae70 0x562677e8c3b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562677ec23d0 0x562677eda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8351ffb700 0 client.5556 ms_handle_reset on v2:172.21.15.3:6833/64074701 2023-11-04T05:29:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562677e7e6f0 0x562677e7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 --2- 172.21.15.3:0/1236968795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562677e7ead0 0x562677e7da40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 >> 172.21.15.3:0/1236968795 conn(0x562677df9660 msgr2=0x562677e7eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 shutdown_connections 2023-11-04T05:29:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.257+0000 7f8385859b80 1 -- 172.21.15.3:0/1236968795 wait complete. 2023-11-04T05:29:49.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f8385859b80 20 client.5556 trim_cache size 0 max 16384 2023-11-04T05:29:49.284 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-11-04T05:29:49.284 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "7a64e299-05b0-4fd0-819f-2262a06846bb", 2023-11-04T05:29:49.285 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-11-04T05:29:49.285 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5379, 2023-11-04T05:29:49.285 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-11-04T05:29:49.285 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-11-04T05:29:49.285 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-11-04T05:29:49.286 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 6.0283711130000004, 2023-11-04T05:29:49.286 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 18, 2023-11-04T05:29:49.286 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 40, 2023-11-04T05:29:49.286 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 40, 2023-11-04T05:29:49.286 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 14.012846007 2023-11-04T05:29:49.287 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-11-04T05:29:49.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f835406af00 msgr2=0x7f835406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f835406af00 0x7f835406d3c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f83740b6360 tx=0x7f8364008030 comp rx=0 tx=0).stop 2023-11-04T05:29:49.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 msgr2=0x7f83740b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740b8710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e840 tx=0x7f835c00ec00 comp rx=0 tx=0).stop 2023-11-04T05:29:49.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 shutdown_connections 2023-11-04T05:29:49.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f835406af00 0x7f835406d3c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f837415ddd0 0x7f83740d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83740d0310 0x7f83740b8710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 --2- 172.21.15.3:0/3803536017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83740d1770 0x7f837414dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 >> 172.21.15.3:0/3803536017 conn(0x7f83740d3900 msgr2=0x7f83740b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 shutdown_connections 2023-11-04T05:29:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.258+0000 7f837965f700 1 -- 172.21.15.3:0/3803536017 wait complete. 2023-11-04T05:29:49.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-04T05:29:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.414+0000 7f8fd3a99700 1 Processor -- start 2023-11-04T05:29:49.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.415+0000 7f8fd3a99700 1 -- start start 2023-11-04T05:29:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.415+0000 7f8fd3a99700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fcc0ccf50 0x7f8fcc0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.415+0000 7f8fd3a99700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc15b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.415+0000 7f8fd3a99700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc15af70 con 0x7f8fcc0cf330 2023-11-04T05:29:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.415+0000 7f8fd3a99700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cde10 con 0x7f8fcc0ccf50 2023-11-04T05:29:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.415+0000 7f8fd3a99700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc072d30 con 0x7f8fcc15b400 2023-11-04T05:29:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1034700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fcc0ccf50 0x7f8fcc0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1835700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8fcc0cf330 0x7f8fcc0cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59512/0 (socket says 172.21.15.3:59512) 2023-11-04T05:29:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1835700 1 -- 172.21.15.3:0/940636702 learned_addr learned my addr 172.21.15.3:0/940636702 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:29:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd2036700 1 --2- 172.21.15.3:0/940636702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1034700 1 -- 172.21.15.3:0/940636702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 msgr2=0x7f8fcc15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:29:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1034700 1 --2- 172.21.15.3:0/940636702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc15b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1034700 1 -- 172.21.15.3:0/940636702 >> v1:172.21.15.3:6789/0 conn(0x7f8fcc0cf330 legacy=0x7f8fcc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1034700 1 -- 172.21.15.3:0/940636702 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0cd570 con 0x7f8fcc0ccf50 2023-11-04T05:29:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd1034700 1 --2- 172.21.15.3:0/940636702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc0d0a40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009820 tx=0x7f8fc0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8167ffbc1fef70a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd0833700 1 -- 172.21.15.3:0/940636702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0005e30 con 0x7f8fcc0ccf50 2023-11-04T05:29:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd0833700 1 -- 172.21.15.3:0/940636702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc0006030 con 0x7f8fcc0ccf50 2023-11-04T05:29:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.416+0000 7f8fd0833700 1 -- 172.21.15.3:0/940636702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00063e0 con 0x7f8fcc0ccf50 2023-11-04T05:29:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 -- 172.21.15.3:0/940636702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0ccf50 msgr2=0x7f8fcc0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/940636702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc0d0a40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009820 tx=0x7f8fc0009b30 comp rx=0 tx=0).stop 2023-11-04T05:29:49.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 -- 172.21.15.3:0/940636702 shutdown_connections 2023-11-04T05:29:49.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/940636702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc15b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/940636702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc0d0a40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 -- 172.21.15.3:0/940636702 >> 172.21.15.3:0/940636702 conn(0x7f8fcc06df90 msgr2=0x7f8fcc06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 -- 172.21.15.3:0/940636702 shutdown_connections 2023-11-04T05:29:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.417+0000 7f8fd3a99700 1 -- 172.21.15.3:0/940636702 wait complete. 2023-11-04T05:29:49.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.418+0000 7f8fd3a99700 1 Processor -- start 2023-11-04T05:29:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.418+0000 7f8fd3a99700 1 -- start start 2023-11-04T05:29:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd3a99700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd3a99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 0x7f8fcc0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd3a99700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd3a99700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1548c0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd3a99700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0b52b0 con 0x7f8fcc0ccf50 2023-11-04T05:29:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd3a99700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0b5590 con 0x7f8fcc15b400 2023-11-04T05:29:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 0x7f8fcc0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 0x7f8fcc0b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54172/0 (socket says 172.21.15.3:54172) 2023-11-04T05:29:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 -- 172.21.15.3:0/3805282642 learned_addr learned my addr 172.21.15.3:0/3805282642 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:29:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd2036700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1835700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 -- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 msgr2=0x7f8fcc159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 -- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fcc0ccf50 msgr2=0x7f8fcc154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 -- 172.21.15.3:0/3805282642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009430 con 0x7f8fcc0cf330 2023-11-04T05:29:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.419+0000 7f8fd1034700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 0x7f8fcc0b20e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009b70 tx=0x7f8fc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.420+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004240 con 0x7f8fcc0cf330 2023-11-04T05:29:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.420+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc00043c0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.420+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc0b5870 con 0x7f8fcc0cf330 2023-11-04T05:29:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.420+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00036a0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.420+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0b5e30 con 0x7f8fcc0cf330 2023-11-04T05:29:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.421+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc0c75f0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.421+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fc0022070 con 0x7f8fcc0cf330 2023-11-04T05:29:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.421+0000 7f8fbe7fc700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8fb006af00 0x7f8fb006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:29:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.422+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8fc008afd0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.422+0000 7f8fd1835700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8fb006af00 0x7f8fb006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:29:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.422+0000 7f8fd1835700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8fb006af00 0x7f8fb006d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005910 tx=0x7f8fc8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:29:49.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.426+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8fc0055fd0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.677+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8fcc074df0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.677+0000 7f8fbe7fc700 1 -- 172.21.15.3:0/3805282642 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8fc00595f0 con 0x7f8fcc0cf330 2023-11-04T05:29:49.680 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-04T05:29:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8fb006af00 msgr2=0x7f8fb006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8fb006af00 0x7f8fb006d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005910 tx=0x7f8fc8009040 comp rx=0 tx=0).stop 2023-11-04T05:29:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 msgr2=0x7f8fcc0b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:29:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 0x7f8fcc0b20e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009b70 tx=0x7f8fc0004750 comp rx=0 tx=0).stop 2023-11-04T05:29:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 shutdown_connections 2023-11-04T05:29:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8fb006af00 0x7f8fb006d3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fcc0ccf50 0x7f8fcc154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fcc15b400 0x7f8fcc159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 --2- 172.21.15.3:0/3805282642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cf330 0x7f8fcc0b20e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:29:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 >> 172.21.15.3:0/3805282642 conn(0x7f8fcc06df90 msgr2=0x7f8fcc15d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:29:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 shutdown_connections 2023-11-04T05:29:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:29:49.679+0000 7f8fd3a99700 1 -- 172.21.15.3:0/3805282642 wait complete. 2023-11-04T05:29:49.693 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-04T05:29:49.694 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-04T05:29:49.694 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-11-04T05:29:49.695 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-04T05:29:49.695 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-04T05:29:49.695 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-04T05:29:49.696 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-04T05:29:49.696 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-04T05:29:49.696 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-04T05:29:49.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:49.697 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-11-04T05:29:49.715 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-04T05:29:49.716 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-04T05:29:49.716 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-11-04T05:29:49.716 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.716 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-11-04T05:29:49.716 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.717 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:49.717 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:49.717 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:49.717 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:49.718 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-04T05:29:49.718 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:49.718 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-04T05:29:49.718 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42316sec preferred_lft 42316sec 2023-11-04T05:29:49.719 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-11-04T05:29:49.719 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.719 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-04T05:29:49.719 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:49.719 INFO:teuthology.orchestra.run.smithi003.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-04T05:29:49.720 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 22:cb:20:1e:85:3a brd ff:ff:ff:ff:ff:ff 2023-11-04T05:29:49.720 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-04T05:29:49.720 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.720 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::8f4:7fff:fea9:c2dc/64 scope link 2023-11-04T05:29:49.720 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.721 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-04T05:29:49.721 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether da:2d:22:80:c4:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-04T05:29:49.721 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::d82d:22ff:fe80:c44a/64 scope link 2023-11-04T05:29:49.721 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.721 INFO:teuthology.orchestra.run.smithi003.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-04T05:29:49.722 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 22:cb:20:1e:85:3a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-04T05:29:49.722 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::20cb:20ff:fe1e:853a/64 scope link 2023-11-04T05:29:49.722 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-11-04T05:29:49.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:49.723 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-04T05:29:49.781 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-11-04T05:29:49.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:49.783 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-11-04T05:29:49.842 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-11-04T05:29:49.842 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-04T05:29:49.843 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-04T05:29:49.843 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-04T05:29:49.843 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-04T05:29:49.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:49.844 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-11-04T05:29:49.844 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-11-04T05:29:49.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-04T05:29:49.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-04T05:29:49.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-04T05:29:49.846 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-11-04T05:29:49.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:49.951 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-11-04T05:29:49.968 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-04T05:29:49.968 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-04T05:29:49.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-04T05:29:49.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:49.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-11-04T05:29:50.036 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:50.037 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-04T05:29:50.094 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-04T05:29:50.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-04T05:29:50.095 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-04T05:29:50.150 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-04T05:29:50.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:50.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:29:50.238 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:29:50.238 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-04T05:29:50.239 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:29:50.247 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:29:50.248 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-04T05:29:50.248 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:29:50.248 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:50.321 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-04T05:29:50.321 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-04T05:29:50.321 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-11-04T05:29:50.322 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-04T05:29:50.322 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-04T05:29:50.322 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790" 2023-11-04T05:29:50.322 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2023-11-04T05:29:50.322 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb" will pass to kernel 2023-11-04T05:29:50.323 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-04T05:29:50.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:50.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-04T05:29:50.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:50.353 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-04T05:29:50.435 INFO:teuthology.orchestra.run.smithi003.stdout:client4931 2023-11-04T05:29:50.436 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:29:50.437 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T05:29:50.494 DEBUG:teuthology.orchestra.run.smithi003:> mktemp -d --suffix=xfstests-dev 2023-11-04T05:29:50.551 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.ByRml5cCmzxfstests-dev 2023-11-04T05:29:50.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:50.552 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.ByRml5cCmzxfstests-dev) 2023-11-04T05:29:50.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:50.619 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.ByRml5cCmzxfstests-dev) 2023-11-04T05:29:50.678 INFO:teuthology.orchestra.run.smithi003.stderr:Cloning into '/tmp/tmp.ByRml5cCmzxfstests-dev'... 2023-11-04T05:29:51.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:51.338 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-11-04T05:29:51.400 DEBUG:teuthology.orchestra.run.smithi003:> mktemp -d --suffix=test 2023-11-04T05:29:51.457 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.Ny4doz2FAUtest 2023-11-04T05:29:51.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:29:51.458 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-11-04T05:29:51.516 DEBUG:teuthology.orchestra.run.smithi003:> mktemp -d --suffix=scratch 2023-11-04T05:29:51.573 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:29:51.574 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "rhel" and "8.6". 2023-11-04T05:29:51.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python3 sqlite xfsprogs-qa-devel 2023-11-04T05:29:51.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:29:51.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:29:51.993 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-11-04T05:29:52.906 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:04:32 ago on Sat 04 Nov 2023 05:25:20 AM UTC. 2023-11-04T05:29:54.343 INFO:teuthology.orchestra.run.smithi003.stdout:Package acl-2.2.53-1.el8.x86_64 is already installed. 2023-11-04T05:29:54.344 INFO:teuthology.orchestra.run.smithi003.stdout:Package attr-2.4.48-3.el8.x86_64 is already installed. 2023-11-04T05:29:54.346 INFO:teuthology.orchestra.run.smithi003.stdout:Package automake-1.16.1-7.el8.noarch is already installed. 2023-11-04T05:29:54.347 INFO:teuthology.orchestra.run.smithi003.stdout:Package bc-1.07.1-5.el8.x86_64 is already installed. 2023-11-04T05:29:54.348 INFO:teuthology.orchestra.run.smithi003.stdout:Package dbench-4.0-20.el8.x86_64 is already installed. 2023-11-04T05:29:54.350 INFO:teuthology.orchestra.run.smithi003.stdout:Package e2fsprogs-1.45.6-4.el8.x86_64 is already installed. 2023-11-04T05:29:54.352 INFO:teuthology.orchestra.run.smithi003.stdout:Package gawk-4.2.1-4.el8.x86_64 is already installed. 2023-11-04T05:29:54.354 INFO:teuthology.orchestra.run.smithi003.stdout:Package gcc-8.5.0-10.1.el8_6.x86_64 is already installed. 2023-11-04T05:29:54.356 INFO:teuthology.orchestra.run.smithi003.stdout:Package libtool-2.4.6-25.el8.x86_64 is already installed. 2023-11-04T05:29:54.357 INFO:teuthology.orchestra.run.smithi003.stdout:Package lvm2-8:2.03.14-3.el8_6.2.x86_64 is already installed. 2023-11-04T05:29:54.358 INFO:teuthology.orchestra.run.smithi003.stdout:Package make-1:4.2.1-11.el8.x86_64 is already installed. 2023-11-04T05:29:54.359 INFO:teuthology.orchestra.run.smithi003.stdout:Package psmisc-23.1-5.el8.x86_64 is already installed. 2023-11-04T05:29:54.361 INFO:teuthology.orchestra.run.smithi003.stdout:Package quota-1:4.04-14.el8.x86_64 is already installed. 2023-11-04T05:29:54.362 INFO:teuthology.orchestra.run.smithi003.stdout:Package sed-4.5-5.el8.x86_64 is already installed. 2023-11-04T05:29:54.363 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-04T05:29:54.364 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-04T05:29:54.365 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-04T05:29:54.366 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-04T05:29:54.368 INFO:teuthology.orchestra.run.smithi003.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2023-11-04T05:29:54.369 INFO:teuthology.orchestra.run.smithi003.stdout:Package libuuid-devel-2.32.1-35.el8.x86_64 is already installed. 2023-11-04T05:29:54.370 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-devel-5.0.0-10.el8.x86_64 is already installed. 2023-11-04T05:29:54.374 INFO:teuthology.orchestra.run.smithi003.stdout:Package python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x86_64 is already installed. 2023-11-04T05:29:54.376 INFO:teuthology.orchestra.run.smithi003.stdout:Package sqlite-3.26.0-15.el8.x86_64 is already installed. 2023-11-04T05:29:54.380 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-04T05:29:54.501 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-11-04T05:29:54.503 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-11-04T05:29:54.503 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-11-04T05:29:54.503 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-11-04T05:29:54.504 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2023-11-04T05:29:54.504 INFO:teuthology.orchestra.run.smithi003.stdout: dump x86_64 1:0.4-0.36.b46.el8 rhel-8-for-x86_64-baseos-rpms 164 k 2023-11-04T05:29:54.504 INFO:teuthology.orchestra.run.smithi003.stdout: fio x86_64 3.19-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-04T05:29:54.505 INFO:teuthology.orchestra.run.smithi003.stdout: indent x86_64 2.2.13-4.el8 epel 187 k 2023-11-04T05:29:54.505 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2023-11-04T05:29:54.506 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-11-04T05:29:54.506 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-11-04T05:29:54.506 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-11-04T05:29:54.507 INFO:teuthology.orchestra.run.smithi003.stdout: daxctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 42 k 2023-11-04T05:29:54.507 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemblk x86_64 1.6.1-1.el8 rhel-8-for-x86_64-appstream-rpms 92 k 2023-11-04T05:29:54.508 INFO:teuthology.orchestra.run.smithi003.stdout: ndctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 79 k 2023-11-04T05:29:54.508 INFO:teuthology.orchestra.run.smithi003.stdout: rmt x86_64 2:1.5.3-13.el8 rhel-8-for-x86_64-baseos-rpms 51 k 2023-11-04T05:29:54.508 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:54.509 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-11-04T05:29:54.509 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-11-04T05:29:54.509 INFO:teuthology.orchestra.run.smithi003.stdout:Install 7 Packages 2023-11-04T05:29:54.510 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 2 Packages 2023-11-04T05:29:54.510 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:29:54.511 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 2.3 M 2023-11-04T05:29:54.512 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-11-04T05:29:54.665 INFO:teuthology.orchestra.run.smithi003.stdout:(1/9): dump-0.4-0.36.b46.el8.x86_64.rpm 1.2 MB/s | 164 kB 00:00 2023-11-04T05:29:54.690 INFO:teuthology.orchestra.run.smithi003.stdout:(2/9): rmt-1.5.3-13.el8.x86_64.rpm 333 kB/s | 51 kB 00:00 2023-11-04T05:29:54.741 INFO:teuthology.orchestra.run.smithi003.stdout:(3/9): daxctl-libs-71.1-3.el8.x86_64.rpm 553 kB/s | 42 kB 00:00 2023-11-04T05:29:54.916 INFO:teuthology.orchestra.run.smithi003.stdout:(4/9): ndctl-libs-71.1-3.el8.x86_64.rpm 349 kB/s | 79 kB 00:00 2023-11-04T05:29:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:(5/9): fio-3.19-3.el8.x86_64.rpm 2.4 MB/s | 538 kB 00:00 2023-11-04T05:29:55.008 INFO:teuthology.orchestra.run.smithi003.stdout:(6/9): libpmemblk-1.6.1-1.el8.x86_64.rpm 1.0 MB/s | 92 kB 00:00 2023-11-04T05:29:55.058 INFO:teuthology.orchestra.run.smithi003.stdout:(7/9): sqlite-3.26.0-16.el8_6.x86_64.rpm 6.5 MB/s | 668 kB 00:00 2023-11-04T05:29:55.133 INFO:teuthology.orchestra.run.smithi003.stdout:(8/9): sqlite-libs-3.26.0-16.el8_6.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2023-11-04T05:29:56.775 INFO:teuthology.orchestra.run.smithi003.stdout:(9/9): indent-2.2.13-4.el8.x86_64.rpm 83 kB/s | 187 kB 00:02 2023-11-04T05:29:56.776 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-11-04T05:29:56.777 INFO:teuthology.orchestra.run.smithi003.stdout:Total 1.0 MB/s | 2.3 MB 00:02 2023-11-04T05:29:56.872 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-11-04T05:29:56.908 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-11-04T05:29:56.908 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-11-04T05:29:57.004 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-11-04T05:29:57.011 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-11-04T05:29:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-11-04T05:29:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : daxctl-libs-71.1-3.el8.x86_64 1/11 2023-11-04T05:29:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: daxctl-libs-71.1-3.el8.x86_64 1/11 2023-11-04T05:29:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ndctl-libs-71.1-3.el8.x86_64 2/11 2023-11-04T05:29:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ndctl-libs-71.1-3.el8.x86_64 2/11 2023-11-04T05:29:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemblk-1.6.1-1.el8.x86_64 3/11 2023-11-04T05:29:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libpmemblk-1.6.1-1.el8.x86_64 3/11 2023-11-04T05:29:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 4/11 2023-11-04T05:29:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rmt-2:1.5.3-13.el8.x86_64 5/11 2023-11-04T05:29:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : dump-1:0.4-0.36.b46.el8.x86_64 6/11 2023-11-04T05:29:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 7/11 2023-11-04T05:29:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fio-3.19-3.el8.x86_64 8/11 2023-11-04T05:29:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : indent-2.2.13-4.el8.x86_64 9/11 2023-11-04T05:29:59.803 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 10/11 2023-11-04T05:29:59.959 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 11/11 2023-11-04T05:30:00.264 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: sqlite-libs-3.26.0-15.el8.x86_64 11/11 2023-11-04T05:30:00.264 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : indent-2.2.13-4.el8.x86_64 1/11 2023-11-04T05:30:00.264 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : dump-1:0.4-0.36.b46.el8.x86_64 2/11 2023-11-04T05:30:00.265 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rmt-2:1.5.3-13.el8.x86_64 3/11 2023-11-04T05:30:00.265 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : daxctl-libs-71.1-3.el8.x86_64 4/11 2023-11-04T05:30:00.265 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ndctl-libs-71.1-3.el8.x86_64 5/11 2023-11-04T05:30:00.265 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fio-3.19-3.el8.x86_64 6/11 2023-11-04T05:30:00.266 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemblk-1.6.1-1.el8.x86_64 7/11 2023-11-04T05:30:00.266 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 8/11 2023-11-04T05:30:00.266 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 9/11 2023-11-04T05:30:00.266 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 10/11 2023-11-04T05:30:01.593 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 11/11 2023-11-04T05:30:01.594 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:04:41 ago on Sat 04 Nov 2023 05:25:20 AM UTC. 2023-11-04T05:30:06.890 INFO:teuthology.orchestra.run.smithi003.stdout:Installed products updated. 2023-11-04T05:30:07.094 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:30:07.095 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2023-11-04T05:30:07.095 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite-3.26.0-16.el8_6.x86_64 sqlite-libs-3.26.0-16.el8_6.x86_64 2023-11-04T05:30:07.095 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-11-04T05:30:07.095 INFO:teuthology.orchestra.run.smithi003.stdout: daxctl-libs-71.1-3.el8.x86_64 dump-1:0.4-0.36.b46.el8.x86_64 2023-11-04T05:30:07.095 INFO:teuthology.orchestra.run.smithi003.stdout: fio-3.19-3.el8.x86_64 indent-2.2.13-4.el8.x86_64 2023-11-04T05:30:07.096 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemblk-1.6.1-1.el8.x86_64 ndctl-libs-71.1-3.el8.x86_64 2023-11-04T05:30:07.096 INFO:teuthology.orchestra.run.smithi003.stdout: rmt-2:1.5.3-13.el8.x86_64 2023-11-04T05:30:07.096 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:30:07.096 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-11-04T05:30:07.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo useradd -m fsgqa 2023-11-04T05:30:07.306 INFO:teuthology.orchestra.run.smithi003.stderr:useradd: user 'fsgqa' already exists 2023-11-04T05:30:07.307 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-11-04T05:30:07.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo groupadd fsgqa 2023-11-04T05:30:07.376 INFO:teuthology.orchestra.run.smithi003.stderr:groupadd: group 'fsgqa' already exists 2023-11-04T05:30:07.377 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-11-04T05:30:07.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo useradd fsgqa2 2023-11-04T05:30:08.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo useradd 123456-fsgqa 2023-11-04T05:30:11.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-11-04T05:30:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.323+0000 7f8a6fb38700 1 Processor -- start 2023-11-04T05:30:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.324+0000 7f8a6fb38700 1 -- start start 2023-11-04T05:30:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.324+0000 7f8a6fb38700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8a680cb150 0x7f8a6815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.324+0000 7f8a6fb38700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6815af80 con 0x7f8a68072650 2023-11-04T05:30:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.324+0000 7f8a6fb38700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6815ea20 con 0x7f8a6815b410 2023-11-04T05:30:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.324+0000 7f8a6fb38700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6815f2c0 con 0x7f8a680cb150 2023-11-04T05:30:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6e0d5700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8a68072650 0x7f8a681510a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59548/0 (socket says 172.21.15.3:59548) 2023-11-04T05:30:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6e0d5700 1 -- 172.21.15.3:0/2900483412 learned_addr learned my addr 172.21.15.3:0/2900483412 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T05:30:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6d8d4700 1 --2- 172.21.15.3:0/2900483412 >> v2:172.21.15.188:3300/0 conn(0x7f8a680cb150 0x7f8a6815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910922290 0 0) 0x7f8a6815af80 con 0x7f8a68072650 2023-11-04T05:30:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a540035b0 con 0x7f8a68072650 2023-11-04T05:30:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555511142 0 0) 0x7f8a6815ea20 con 0x7f8a6815b410 2023-11-04T05:30:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a6815af80 con 0x7f8a6815b410 2023-11-04T05:30:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508107027 0 0) 0x7f8a540035b0 con 0x7f8a68072650 2023-11-04T05:30:11.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 >> v2:172.21.15.188:3300/0 conn(0x7f8a680cb150 msgr2=0x7f8a6815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:30:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 --2- 172.21.15.3:0/2900483412 >> v2:172.21.15.188:3300/0 conn(0x7f8a680cb150 0x7f8a6815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 >> v1:172.21.15.188:6789/0 conn(0x7f8a6815b410 legacy=0x7f8a68071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6815ed00 con 0x7f8a68072650 2023-11-04T05:30:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2657708889 0 0) 0x7f8a600019f0 con 0x7f8a68072650 2023-11-04T05:30:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 >> v1:172.21.15.3:6789/0 conn(0x7f8a68072650 legacy=0x7f8a681510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54003be0 0x7f8a54003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a54004850 0x7f8a54004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54005210 0x7f8a5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6815f2c0 con 0x7f8a54004850 2023-11-04T05:30:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5400c0e0 con 0x7f8a54003be0 2023-11-04T05:30:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5400c2a0 con 0x7f8a54005210 2023-11-04T05:30:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6d0d3700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a54004850 0x7f8a54004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.325+0000 7f8a6d8d4700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54003be0 0x7f8a54003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6e0d5700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54005210 0x7f8a5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6d0d3700 1 -- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54005210 msgr2=0x7f8a5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:30:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6d0d3700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54005210 0x7f8a5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6d0d3700 1 -- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54003be0 msgr2=0x7f8a54003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:30:11.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6d0d3700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54003be0 0x7f8a54003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6e0d5700 1 -- 172.21.15.3:0/2900483412 reap_dead start 2023-11-04T05:30:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6d0d3700 1 -- 172.21.15.3:0/2900483412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540035b0 con 0x7f8a54004850 2023-11-04T05:30:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6d0d3700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a54004850 0x7f8a54004cd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001940 tx=0x7f8a54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85ad0c7a3192a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:30:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c001430 con 0x7f8a54004850 2023-11-04T05:30:11.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c001d70 con 0x7f8a54004850 2023-11-04T05:30:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.326+0000 7f8a6c8d2700 1 -- 172.21.15.3:0/2900483412 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c005510 con 0x7f8a54004850 2023-11-04T05:30:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 -- 172.21.15.3:0/2900483412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a54004850 msgr2=0x7f8a54004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a54004850 0x7f8a54004cd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001940 tx=0x7f8a54005880 comp rx=0 tx=0).stop 2023-11-04T05:30:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 -- 172.21.15.3:0/2900483412 shutdown_connections 2023-11-04T05:30:11.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/2900483412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a54004850 0x7f8a54004cd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 -- 172.21.15.3:0/2900483412 >> 172.21.15.3:0/2900483412 conn(0x7f8a6806b150 msgr2=0x7f8a6806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:30:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 -- 172.21.15.3:0/2900483412 shutdown_connections 2023-11-04T05:30:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.327+0000 7f8a6fb38700 1 -- 172.21.15.3:0/2900483412 wait complete. 2023-11-04T05:30:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.328+0000 7f8a6fb38700 1 Processor -- start 2023-11-04T05:30:11.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.328+0000 7f8a6fb38700 1 -- start start 2023-11-04T05:30:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6fb38700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54004850 0x7f8a681543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6fb38700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a681f3bd0 0x7f8a680cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6fb38700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 0x7f8a680b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6fb38700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a681f3740 con 0x7f8a681f4050 2023-11-04T05:30:11.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6fb38700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68154920 con 0x7f8a681f3bd0 2023-11-04T05:30:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6fb38700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a681f3410 con 0x7f8a54004850 2023-11-04T05:30:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 0x7f8a680b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 0x7f8a680b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54212/0 (socket says 172.21.15.3:54212) 2023-11-04T05:30:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 -- 172.21.15.3:0/3771780260 learned_addr learned my addr 172.21.15.3:0/3771780260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:30:11.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6d0d3700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a681f3bd0 0x7f8a680cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6d8d4700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54004850 0x7f8a681543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 -- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54004850 msgr2=0x7f8a681543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54004850 0x7f8a681543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 -- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a681f3bd0 msgr2=0x7f8a680cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a681f3bd0 0x7f8a680cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 -- 172.21.15.3:0/3771780260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680c9fc0 con 0x7f8a681f4050 2023-11-04T05:30:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6d0d3700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a681f3bd0 0x7f8a680cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:30:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6d8d4700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54004850 0x7f8a681543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-04T05:30:11.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a6e0d5700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 0x7f8a680b01c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a60002230 tx=0x7f8a6000a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:30:11.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000ace0 con 0x7f8a681f4050 2023-11-04T05:30:11.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.329+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000ae40 con 0x7f8a681f4050 2023-11-04T05:30:11.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.330+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a680ca2a0 con 0x7f8a681f4050 2023-11-04T05:30:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.330+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60013630 con 0x7f8a681f4050 2023-11-04T05:30:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.330+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a68159c90 con 0x7f8a681f4050 2023-11-04T05:30:11.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.331+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68074df0 con 0x7f8a681f4050 2023-11-04T05:30:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.331+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a60013790 con 0x7f8a681f4050 2023-11-04T05:30:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.331+0000 7f8a5a7fc700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8a4c06af00 0x7f8a4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.332+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a6008a230 con 0x7f8a681f4050 2023-11-04T05:30:11.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.332+0000 7f8a6d8d4700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8a4c06af00 0x7f8a4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.332+0000 7f8a6d8d4700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8a4c06af00 0x7f8a4c06d3c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009a80 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:30:11.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.336+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f8a60058ca0 con 0x7f8a681f4050 2023-11-04T05:30:11.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.624+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8a681f3880 con 0x7f8a681f4050 2023-11-04T05:30:11.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.624+0000 7f8a5a7fc700 1 -- 172.21.15.3:0/3771780260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1272 (secure 0 0 0) 0x7f8a60017510 con 0x7f8a681f4050 2023-11-04T05:30:11.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:30:11.627 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","modified":"2023-11-04T05:27:23.271086Z","created":"2023-11-04T05:27:23.271086Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3301","nonce":0},{"type":"v1","addr":"172.21.15.188:6790","nonce":0}]},"addr":"172.21.15.188:6790/0","public_addr":"172.21.15.188:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-11-04T05:30:11.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8a4c06af00 msgr2=0x7f8a4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8a4c06af00 0x7f8a4c06d3c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009a80 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2023-11-04T05:30:11.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 msgr2=0x7f8a680b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 0x7f8a680b01c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a60002230 tx=0x7f8a6000a510 comp rx=0 tx=0).stop 2023-11-04T05:30:11.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 shutdown_connections 2023-11-04T05:30:11.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f8a4c06af00 0x7f8a4c06d3c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a681f4050 0x7f8a680b01c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a54004850 0x7f8a681543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 --2- 172.21.15.3:0/3771780260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a681f3bd0 0x7f8a680cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 >> 172.21.15.3:0/3771780260 conn(0x7f8a6806b150 msgr2=0x7f8a680c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:30:11.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 shutdown_connections 2023-11-04T05:30:11.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.626+0000 7f8a6fb38700 1 -- 172.21.15.3:0/3771780260 wait complete. 2023-11-04T05:30:11.631 INFO:teuthology.orchestra.run.smithi003.stderr:dumped monmap epoch 1 2023-11-04T05:30:11.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2023-11-04T05:30:11.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.765+0000 7fcf08f19700 1 Processor -- start 2023-11-04T05:30:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.765+0000 7fcf08f19700 1 -- start start 2023-11-04T05:30:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf08f19700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf0415bcc0 0x7fcf040ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf08f19700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf040d0400 0x7fcf040aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf08f19700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf040aead0 con 0x7fcf040ce630 2023-11-04T05:30:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf08f19700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040cee20 con 0x7fcf0415bcc0 2023-11-04T05:30:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf08f19700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040ceb40 con 0x7fcf040d0400 2023-11-04T05:30:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf0259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf0415bcc0 0x7fcf040ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf0259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf0415bcc0 0x7fcf040ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54216/0 (socket says 172.21.15.3:54216) 2023-11-04T05:30:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.766+0000 7fcf0259c700 1 -- 172.21.15.3:0/2773702618 learned_addr learned my addr 172.21.15.3:0/2773702618 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:30:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0259c700 1 -- 172.21.15.3:0/2773702618 >> v2:172.21.15.188:3300/0 conn(0x7fcf040d0400 msgr2=0x7fcf040aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:30:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf02d9d700 1 --2- 172.21.15.3:0/2773702618 >> v2:172.21.15.188:3300/0 conn(0x7fcf040d0400 0x7fcf040aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0259c700 1 --2- 172.21.15.3:0/2773702618 >> v2:172.21.15.188:3300/0 conn(0x7fcf040d0400 0x7fcf040aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0259c700 1 -- 172.21.15.3:0/2773702618 >> v1:172.21.15.188:6789/0 conn(0x7fcf040ce630 legacy=0x7fcf040cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T05:30:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0259c700 1 -- 172.21.15.3:0/2773702618 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf040ae480 con 0x7fcf0415bcc0 2023-11-04T05:30:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0259c700 1 --2- 172.21.15.3:0/2773702618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf0415bcc0 0x7fcf040ce0f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcf040d2810 tx=0x7fcef4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbd20dd607ec2a server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:30:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0159a700 1 -- 172.21.15.3:0/2773702618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4004230 con 0x7fcf0415bcc0 2023-11-04T05:30:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0159a700 1 -- 172.21.15.3:0/2773702618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef4004390 con 0x7fcf0415bcc0 2023-11-04T05:30:11.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.767+0000 7fcf0159a700 1 -- 172.21.15.3:0/2773702618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40036a0 con 0x7fcf0415bcc0 2023-11-04T05:30:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 -- 172.21.15.3:0/2773702618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf0415bcc0 msgr2=0x7fcf040ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 --2- 172.21.15.3:0/2773702618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf0415bcc0 0x7fcf040ce0f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcf040d2810 tx=0x7fcef4009bd0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 -- 172.21.15.3:0/2773702618 shutdown_connections 2023-11-04T05:30:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 --2- 172.21.15.3:0/2773702618 >> v2:172.21.15.188:3300/0 conn(0x7fcf040d0400 0x7fcf040aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 --2- 172.21.15.3:0/2773702618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf0415bcc0 0x7fcf040ce0f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 -- 172.21.15.3:0/2773702618 >> 172.21.15.3:0/2773702618 conn(0x7fcf0409a9f0 msgr2=0x7fcf040d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:30:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 -- 172.21.15.3:0/2773702618 shutdown_connections 2023-11-04T05:30:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.768+0000 7fcf08f19700 1 -- 172.21.15.3:0/2773702618 wait complete. 2023-11-04T05:30:11.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.769+0000 7fcf08f19700 1 Processor -- start 2023-11-04T05:30:11.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.770+0000 7fcf08f19700 1 -- start start 2023-11-04T05:30:11.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.770+0000 7fcf08f19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 0x7fcf041595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.770+0000 7fcf08f19700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf040d0400 0x7fcf040b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf08f19700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0415bcc0 0x7fcf040b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf08f19700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04159af0 con 0x7fcf040ce630 2023-11-04T05:30:11.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf08f19700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040b5b90 con 0x7fcf0415bcc0 2023-11-04T05:30:11.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf08f19700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040b52f0 con 0x7fcf040d0400 2023-11-04T05:30:11.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 0x7fcf041595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 0x7fcf041595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54220/0 (socket says 172.21.15.3:54220) 2023-11-04T05:30:11.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 -- 172.21.15.3:0/3798088252 learned_addr learned my addr 172.21.15.3:0/3798088252 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T05:30:11.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf01d9b700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf040d0400 0x7fcf040b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf02d9d700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0415bcc0 0x7fcf040b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 -- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf040d0400 msgr2=0x7fcf040b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf040d0400 0x7fcf040b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 -- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0415bcc0 msgr2=0x7fcf040b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:11.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0415bcc0 0x7fcf040b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:11.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 -- 172.21.15.3:0/3798088252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4009550 con 0x7fcf040ce630 2023-11-04T05:30:11.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf0259c700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 0x7fcf041595b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009130 tx=0x7fcef4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:30:11.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4011630 con 0x7fcf040ce630 2023-11-04T05:30:11.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef4011790 con 0x7fcf040ce630 2023-11-04T05:30:11.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf040b55d0 con 0x7fcf040ce630 2023-11-04T05:30:11.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4011b00 con 0x7fcf040ce630 2023-11-04T05:30:11.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.771+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf040b24b0 con 0x7fcf040ce630 2023-11-04T05:30:11.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.772+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf040b2310 con 0x7fcf040ce630 2023-11-04T05:30:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.773+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcef4011c60 con 0x7fcf040ce630 2023-11-04T05:30:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.773+0000 7fcef2ffd700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcee406af00 0x7fcee406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T05:30:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.773+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcef408ae00 con 0x7fcf040ce630 2023-11-04T05:30:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.773+0000 7fcf01d9b700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcee406af00 0x7fcee406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T05:30:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.774+0000 7fcf01d9b700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcee406af00 0x7fcee406d3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcf040c5160 tx=0x7fcef8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T05:30:11.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:11.778+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fcf040b2310 con 0x7fcf040ce630 2023-11-04T05:30:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.064+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7fcf040b2310 con 0x7fcf040ce630 2023-11-04T05:30:12.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.064+0000 7fcef2ffd700 1 -- 172.21.15.3:0/3798088252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v9) v1 ==== 92+0+63 (secure 0 0 0) 0x7fcf040b2310 con 0x7fcf040ce630 2023-11-04T05:30:12.067 INFO:teuthology.orchestra.run.smithi003.stdout:[client.admin] 2023-11-04T05:30:12.067 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== 2023-11-04T05:30:12.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcee406af00 msgr2=0x7fcee406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:12.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcee406af00 0x7fcee406d3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcf040c5160 tx=0x7fcef8009040 comp rx=0 tx=0).stop 2023-11-04T05:30:12.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 msgr2=0x7fcf041595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T05:30:12.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 0x7fcf041595b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009130 tx=0x7fcef4004750 comp rx=0 tx=0).stop 2023-11-04T05:30:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 shutdown_connections 2023-11-04T05:30:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fcee406af00 0x7fcee406d3c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0415bcc0 0x7fcf040b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf040ce630 0x7fcf041595b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 --2- 172.21.15.3:0/3798088252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf040d0400 0x7fcf040b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T05:30:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 >> 172.21.15.3:0/3798088252 conn(0x7fcf0409a9f0 msgr2=0x7fcf040c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T05:30:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 shutdown_connections 2023-11-04T05:30:12.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T05:30:12.066+0000 7fcf08f19700 1 -- 172.21.15.3:0/3798088252 wait complete. 2023-11-04T05:30:12.081 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:30:12.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/tmp/tmp.ByRml5cCmzxfstests-dev/local.config 2023-11-04T05:30:12.108 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.3:6789:/test export TEST_DIR=/tmp/tmp.Ny4doz2FAUtest export SCRATCH_DEV=172.21.15.3:6789:/scratch export SCRATCH_MNT=/tmp/tmp.WC1AnNZjMJscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg==" 2023-11-04T05:30:12.109 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T05:30:12.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/tmp/tmp.ByRml5cCmzxfstests-dev/ceph.exclude 2023-11-04T05:30:12.176 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo make) 2023-11-04T05:30:12.258 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize -cfi 2023-11-04T05:30:12.585 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: putting auxiliary files in '.'. 2023-11-04T05:30:12.586 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file './config.guess' 2023-11-04T05:30:12.589 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file './config.sub' 2023-11-04T05:30:12.592 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file './install-sh' 2023-11-04T05:30:12.596 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file './ltmain.sh' 2023-11-04T05:30:12.617 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-11-04T05:30:12.618 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-11-04T05:30:12.622 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-11-04T05:30:12.626 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-11-04T05:30:12.631 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-11-04T05:30:12.635 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-11-04T05:30:12.641 INFO:teuthology.orchestra.run.smithi003.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-11-04T05:30:12.642 INFO:teuthology.orchestra.run.smithi003.stdout:cp include/install-sh . 2023-11-04T05:30:12.643 INFO:teuthology.orchestra.run.smithi003.stdout:aclocal -I m4 2023-11-04T05:30:13.816 INFO:teuthology.orchestra.run.smithi003.stdout:autoheader 2023-11-04T05:30:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:autoconf 2023-11-04T05:30:14.305 INFO:teuthology.orchestra.run.smithi003.stdout:./configure \ 2023-11-04T05:30:14.305 INFO:teuthology.orchestra.run.smithi003.stdout: --libexecdir=/usr/lib \ 2023-11-04T05:30:14.305 INFO:teuthology.orchestra.run.smithi003.stdout: --exec_prefix=/var/lib 2023-11-04T05:30:14.781 INFO:teuthology.orchestra.run.smithi003.stdout:checking build system type... x86_64-pc-linux-gnu 2023-11-04T05:30:14.782 INFO:teuthology.orchestra.run.smithi003.stdout:checking host system type... x86_64-pc-linux-gnu 2023-11-04T05:30:14.783 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to print strings... printf 2023-11-04T05:30:14.783 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gcc... gcc 2023-11-04T05:30:14.904 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether the C compiler works... yes 2023-11-04T05:30:14.905 INFO:teuthology.orchestra.run.smithi003.stdout:checking for C compiler default output file name... a.out 2023-11-04T05:30:14.933 INFO:teuthology.orchestra.run.smithi003.stdout:checking for suffix of executables... 2023-11-04T05:30:14.982 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether we are cross compiling... no 2023-11-04T05:30:15.000 INFO:teuthology.orchestra.run.smithi003.stdout:checking for suffix of object files... o 2023-11-04T05:30:15.017 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether we are using the GNU C compiler... yes 2023-11-04T05:30:15.034 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether gcc accepts -g... yes 2023-11-04T05:30:15.060 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gcc option to accept ISO C89... none needed 2023-11-04T05:30:15.063 INFO:teuthology.orchestra.run.smithi003.stdout:checking for a sed that does not truncate output... /bin/sed 2023-11-04T05:30:15.065 INFO:teuthology.orchestra.run.smithi003.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-11-04T05:30:15.066 INFO:teuthology.orchestra.run.smithi003.stdout:checking for egrep... /bin/grep -E 2023-11-04T05:30:15.067 INFO:teuthology.orchestra.run.smithi003.stdout:checking for fgrep... /bin/grep -F 2023-11-04T05:30:15.071 INFO:teuthology.orchestra.run.smithi003.stdout:checking for ld used by gcc... /bin/ld 2023-11-04T05:30:15.072 INFO:teuthology.orchestra.run.smithi003.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-11-04T05:30:15.078 INFO:teuthology.orchestra.run.smithi003.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-11-04T05:30:15.094 INFO:teuthology.orchestra.run.smithi003.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-11-04T05:30:15.094 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether ln -s works... yes 2023-11-04T05:30:15.099 INFO:teuthology.orchestra.run.smithi003.stdout:checking the maximum length of command line arguments... 1572864 2023-11-04T05:30:15.102 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-11-04T05:30:15.102 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-11-04T05:30:15.102 INFO:teuthology.orchestra.run.smithi003.stdout:checking for /bin/ld option to reload object files... -r 2023-11-04T05:30:15.103 INFO:teuthology.orchestra.run.smithi003.stdout:checking for objdump... objdump 2023-11-04T05:30:15.104 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to recognize dependent libraries... pass_all 2023-11-04T05:30:15.104 INFO:teuthology.orchestra.run.smithi003.stdout:checking for dlltool... no 2023-11-04T05:30:15.104 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-11-04T05:30:15.104 INFO:teuthology.orchestra.run.smithi003.stdout:checking for ar... ar 2023-11-04T05:30:15.131 INFO:teuthology.orchestra.run.smithi003.stdout:checking for archiver @FILE support... @ 2023-11-04T05:30:15.131 INFO:teuthology.orchestra.run.smithi003.stdout:checking for strip... strip 2023-11-04T05:30:15.131 INFO:teuthology.orchestra.run.smithi003.stdout:checking for ranlib... ranlib 2023-11-04T05:30:15.132 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gawk... gawk 2023-11-04T05:30:15.183 INFO:teuthology.orchestra.run.smithi003.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-11-04T05:30:15.184 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sysroot... no 2023-11-04T05:30:15.187 INFO:teuthology.orchestra.run.smithi003.stdout:checking for a working dd... /bin/dd 2023-11-04T05:30:15.189 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-11-04T05:30:15.202 INFO:teuthology.orchestra.run.smithi003.stdout:checking for mt... no 2023-11-04T05:30:15.204 INFO:teuthology.orchestra.run.smithi003.stdout:checking if : is a manifest tool... no 2023-11-04T05:30:15.248 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to run the C preprocessor... gcc -E 2023-11-04T05:30:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:checking for ANSI C header files... yes 2023-11-04T05:30:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/types.h... yes 2023-11-04T05:30:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/stat.h... yes 2023-11-04T05:30:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:checking for stdlib.h... yes 2023-11-04T05:30:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:checking for string.h... yes 2023-11-04T05:30:15.525 INFO:teuthology.orchestra.run.smithi003.stdout:checking for memory.h... yes 2023-11-04T05:30:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:checking for strings.h... yes 2023-11-04T05:30:15.583 INFO:teuthology.orchestra.run.smithi003.stdout:checking for inttypes.h... yes 2023-11-04T05:30:15.608 INFO:teuthology.orchestra.run.smithi003.stdout:checking for stdint.h... yes 2023-11-04T05:30:15.638 INFO:teuthology.orchestra.run.smithi003.stdout:checking for unistd.h... yes 2023-11-04T05:30:15.682 INFO:teuthology.orchestra.run.smithi003.stdout:checking for dlfcn.h... yes 2023-11-04T05:30:15.685 INFO:teuthology.orchestra.run.smithi003.stdout:checking for objdir... .libs 2023-11-04T05:30:15.752 INFO:teuthology.orchestra.run.smithi003.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-11-04T05:30:15.752 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-11-04T05:30:15.767 INFO:teuthology.orchestra.run.smithi003.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-11-04T05:30:15.787 INFO:teuthology.orchestra.run.smithi003.stdout:checking if gcc static flag -static works... no 2023-11-04T05:30:15.808 INFO:teuthology.orchestra.run.smithi003.stdout:checking if gcc supports -c -o file.o... yes 2023-11-04T05:30:15.809 INFO:teuthology.orchestra.run.smithi003.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-11-04T05:30:15.814 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-11-04T05:30:15.839 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether -lc should be explicitly linked in... no 2023-11-04T05:30:15.883 INFO:teuthology.orchestra.run.smithi003.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-11-04T05:30:15.883 INFO:teuthology.orchestra.run.smithi003.stdout:checking how to hardcode library paths into programs... immediate 2023-11-04T05:30:15.886 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether stripping libraries is possible... yes 2023-11-04T05:30:15.886 INFO:teuthology.orchestra.run.smithi003.stdout:checking if libtool supports shared libraries... yes 2023-11-04T05:30:15.886 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether to build shared libraries... yes 2023-11-04T05:30:15.886 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether to build static libraries... yes 2023-11-04T05:30:15.890 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gcc... (cached) gcc 2023-11-04T05:30:15.906 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-11-04T05:30:15.906 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether gcc accepts -g... (cached) yes 2023-11-04T05:30:15.907 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-11-04T05:30:15.907 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gmake... /usr/bin/gmake 2023-11-04T05:30:15.907 INFO:teuthology.orchestra.run.smithi003.stderr:./configure: line 12070: test: $(SHELL): binary operator expected 2023-11-04T05:30:15.908 INFO:teuthology.orchestra.run.smithi003.stdout:checking for tar... /bin/tar 2023-11-04T05:30:15.908 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gzip... /bin/gzip 2023-11-04T05:30:15.914 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether gcc -MM is supported... yes 2023-11-04T05:30:15.915 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sort... /bin/sort 2023-11-04T05:30:15.915 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether ln -s works... yes 2023-11-04T05:30:15.915 INFO:teuthology.orchestra.run.smithi003.stdout:checking for rpm... /bin/rpm 2023-11-04T05:30:15.919 INFO:teuthology.orchestra.run.smithi003.stdout:checking for rpmbuild... no 2023-11-04T05:30:15.944 INFO:teuthology.orchestra.run.smithi003.stdout:checking assert.h usability... yes 2023-11-04T05:30:15.951 INFO:teuthology.orchestra.run.smithi003.stdout:checking assert.h presence... yes 2023-11-04T05:30:15.951 INFO:teuthology.orchestra.run.smithi003.stdout:checking for assert.h... yes 2023-11-04T05:30:15.977 INFO:teuthology.orchestra.run.smithi003.stdout:checking bstring.h usability... no 2023-11-04T05:30:15.987 INFO:teuthology.orchestra.run.smithi003.stdout:checking bstring.h presence... no 2023-11-04T05:30:15.987 INFO:teuthology.orchestra.run.smithi003.stdout:checking for bstring.h... no 2023-11-04T05:30:16.012 INFO:teuthology.orchestra.run.smithi003.stdout:checking libgen.h usability... yes 2023-11-04T05:30:16.018 INFO:teuthology.orchestra.run.smithi003.stdout:checking libgen.h presence... yes 2023-11-04T05:30:16.019 INFO:teuthology.orchestra.run.smithi003.stdout:checking for libgen.h... yes 2023-11-04T05:30:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:checking dirent.h usability... yes 2023-11-04T05:30:16.055 INFO:teuthology.orchestra.run.smithi003.stdout:checking dirent.h presence... yes 2023-11-04T05:30:16.055 INFO:teuthology.orchestra.run.smithi003.stdout:checking for dirent.h... yes 2023-11-04T05:30:16.118 INFO:teuthology.orchestra.run.smithi003.stdout:checking errno.h usability... yes 2023-11-04T05:30:16.125 INFO:teuthology.orchestra.run.smithi003.stdout:checking errno.h presence... yes 2023-11-04T05:30:16.125 INFO:teuthology.orchestra.run.smithi003.stdout:checking for errno.h... yes 2023-11-04T05:30:16.153 INFO:teuthology.orchestra.run.smithi003.stdout:checking malloc.h usability... yes 2023-11-04T05:30:16.161 INFO:teuthology.orchestra.run.smithi003.stdout:checking malloc.h presence... yes 2023-11-04T05:30:16.161 INFO:teuthology.orchestra.run.smithi003.stdout:checking for malloc.h... yes 2023-11-04T05:30:16.189 INFO:teuthology.orchestra.run.smithi003.stdout:checking uuid.h usability... no 2023-11-04T05:30:16.198 INFO:teuthology.orchestra.run.smithi003.stdout:checking uuid.h presence... no 2023-11-04T05:30:16.199 INFO:teuthology.orchestra.run.smithi003.stdout:checking for uuid.h... no 2023-11-04T05:30:16.225 INFO:teuthology.orchestra.run.smithi003.stdout:checking uuid/uuid.h usability... yes 2023-11-04T05:30:16.234 INFO:teuthology.orchestra.run.smithi003.stdout:checking uuid/uuid.h presence... yes 2023-11-04T05:30:16.234 INFO:teuthology.orchestra.run.smithi003.stdout:checking for uuid/uuid.h... yes 2023-11-04T05:30:16.260 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/uuid.h usability... no 2023-11-04T05:30:16.270 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/uuid.h presence... no 2023-11-04T05:30:16.270 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/uuid.h... no 2023-11-04T05:30:16.301 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/file.h usability... yes 2023-11-04T05:30:16.309 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/file.h presence... yes 2023-11-04T05:30:16.309 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/file.h... yes 2023-11-04T05:30:16.337 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/fcntl.h usability... yes 2023-11-04T05:30:16.344 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/fcntl.h presence... yes 2023-11-04T05:30:16.344 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/fcntl.h... yes 2023-11-04T05:30:16.370 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/syssgi.h usability... no 2023-11-04T05:30:16.380 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/syssgi.h presence... no 2023-11-04T05:30:16.380 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/syssgi.h... no 2023-11-04T05:30:16.418 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/param.h usability... yes 2023-11-04T05:30:16.428 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/param.h presence... yes 2023-11-04T05:30:16.429 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/param.h... yes 2023-11-04T05:30:16.434 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/stat.h... (cached) yes 2023-11-04T05:30:16.461 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/statvfs.h usability... yes 2023-11-04T05:30:16.468 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/statvfs.h presence... yes 2023-11-04T05:30:16.469 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/statvfs.h... yes 2023-11-04T05:30:16.496 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/time.h usability... yes 2023-11-04T05:30:16.503 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/time.h presence... yes 2023-11-04T05:30:16.503 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/time.h... yes 2023-11-04T05:30:16.533 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/ioctl.h usability... yes 2023-11-04T05:30:16.540 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/ioctl.h presence... yes 2023-11-04T05:30:16.541 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/ioctl.h... yes 2023-11-04T05:30:16.570 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/wait.h usability... yes 2023-11-04T05:30:16.579 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/wait.h presence... yes 2023-11-04T05:30:16.579 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/wait.h... yes 2023-11-04T05:30:16.584 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/types.h... (cached) yes 2023-11-04T05:30:16.589 INFO:teuthology.orchestra.run.smithi003.stdout:checking for strings.h... (cached) yes 2023-11-04T05:30:16.617 INFO:teuthology.orchestra.run.smithi003.stdout:checking err.h usability... yes 2023-11-04T05:30:16.624 INFO:teuthology.orchestra.run.smithi003.stdout:checking err.h presence... yes 2023-11-04T05:30:16.624 INFO:teuthology.orchestra.run.smithi003.stdout:checking for err.h... yes 2023-11-04T05:30:16.652 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/falloc.h usability... yes 2023-11-04T05:30:16.658 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/falloc.h presence... yes 2023-11-04T05:30:16.658 INFO:teuthology.orchestra.run.smithi003.stdout:checking for linux/falloc.h... yes 2023-11-04T05:30:16.684 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-11-04T05:30:16.694 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-11-04T05:30:16.694 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/fs/xfs_fsops.h... no 2023-11-04T05:30:16.717 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/fs/xfs_itable.h usability... no 2023-11-04T05:30:16.727 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/fs/xfs_itable.h presence... no 2023-11-04T05:30:16.727 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/fs/xfs_itable.h... no 2023-11-04T05:30:16.750 INFO:teuthology.orchestra.run.smithi003.stdout:checking xfs/platform_defs.h usability... no 2023-11-04T05:30:16.760 INFO:teuthology.orchestra.run.smithi003.stdout:checking xfs/platform_defs.h presence... no 2023-11-04T05:30:16.760 INFO:teuthology.orchestra.run.smithi003.stdout:checking for xfs/platform_defs.h... no 2023-11-04T05:30:16.783 INFO:teuthology.orchestra.run.smithi003.stdout:checking btrfs/ioctl.h usability... no 2023-11-04T05:30:16.793 INFO:teuthology.orchestra.run.smithi003.stdout:checking btrfs/ioctl.h presence... no 2023-11-04T05:30:16.793 INFO:teuthology.orchestra.run.smithi003.stdout:checking for btrfs/ioctl.h... no 2023-11-04T05:30:16.816 INFO:teuthology.orchestra.run.smithi003.stdout:checking cifs/ioctl.h usability... no 2023-11-04T05:30:16.825 INFO:teuthology.orchestra.run.smithi003.stdout:checking cifs/ioctl.h presence... no 2023-11-04T05:30:16.826 INFO:teuthology.orchestra.run.smithi003.stdout:checking for cifs/ioctl.h... no 2023-11-04T05:30:16.851 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/mman.h usability... yes 2023-11-04T05:30:16.859 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/mman.h presence... yes 2023-11-04T05:30:16.859 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/mman.h... yes 2023-11-04T05:30:16.874 INFO:teuthology.orchestra.run.smithi003.stdout:checking for xfs/xfs_log_format.h... no 2023-11-04T05:30:16.908 INFO:teuthology.orchestra.run.smithi003.stdout:checking for uuid_compare... no 2023-11-04T05:30:16.939 INFO:teuthology.orchestra.run.smithi003.stdout:checking for uuid_compare in -luuid... yes 2023-11-04T05:30:17.004 INFO:teuthology.orchestra.run.smithi003.stdout:checking for xfs/xfs.h... yes 2023-11-04T05:30:17.018 INFO:teuthology.orchestra.run.smithi003.stdout:checking for xfs/libxfs.h... no 2023-11-04T05:30:17.034 INFO:teuthology.orchestra.run.smithi003.stdout:checking whether xlog_assign_lsn is declared... no 2023-11-04T05:30:17.061 INFO:teuthology.orchestra.run.smithi003.stdout:checking for xfs/xqm.h... yes 2023-11-04T05:30:17.108 INFO:teuthology.orchestra.run.smithi003.stdout:checking xfsctl from xfs/xfs.h... ok 2023-11-04T05:30:17.131 INFO:teuthology.orchestra.run.smithi003.stdout:checking xfs/handle.h usability... yes 2023-11-04T05:30:17.137 INFO:teuthology.orchestra.run.smithi003.stdout:checking xfs/handle.h presence... yes 2023-11-04T05:30:17.137 INFO:teuthology.orchestra.run.smithi003.stdout:checking for xfs/handle.h... yes 2023-11-04T05:30:17.179 INFO:teuthology.orchestra.run.smithi003.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-11-04T05:30:17.207 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/acl.h usability... yes 2023-11-04T05:30:17.215 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/acl.h presence... yes 2023-11-04T05:30:17.216 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/acl.h... yes 2023-11-04T05:30:17.243 INFO:teuthology.orchestra.run.smithi003.stdout:checking acl/libacl.h usability... yes 2023-11-04T05:30:17.251 INFO:teuthology.orchestra.run.smithi003.stdout:checking acl/libacl.h presence... yes 2023-11-04T05:30:17.252 INFO:teuthology.orchestra.run.smithi003.stdout:checking for acl/libacl.h... yes 2023-11-04T05:30:17.284 INFO:teuthology.orchestra.run.smithi003.stdout:checking for acl_init in -lacl... yes 2023-11-04T05:30:17.306 INFO:teuthology.orchestra.run.smithi003.stdout:checking gdbm-ndbm.h usability... no 2023-11-04T05:30:17.316 INFO:teuthology.orchestra.run.smithi003.stdout:checking gdbm-ndbm.h presence... no 2023-11-04T05:30:17.316 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gdbm-ndbm.h... no 2023-11-04T05:30:17.337 INFO:teuthology.orchestra.run.smithi003.stdout:checking gdbm/ndbm.h usability... no 2023-11-04T05:30:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:checking gdbm/ndbm.h presence... no 2023-11-04T05:30:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:checking for gdbm/ndbm.h... no 2023-11-04T05:30:17.369 INFO:teuthology.orchestra.run.smithi003.stdout:checking ndbm.h usability... no 2023-11-04T05:30:17.378 INFO:teuthology.orchestra.run.smithi003.stdout:checking ndbm.h presence... no 2023-11-04T05:30:17.379 INFO:teuthology.orchestra.run.smithi003.stdout:checking for ndbm.h... no 2023-11-04T05:30:17.404 INFO:teuthology.orchestra.run.smithi003.stdout:checking libaio.h usability... yes 2023-11-04T05:30:17.414 INFO:teuthology.orchestra.run.smithi003.stdout:checking libaio.h presence... yes 2023-11-04T05:30:17.415 INFO:teuthology.orchestra.run.smithi003.stdout:checking for libaio.h... yes 2023-11-04T05:30:17.437 INFO:teuthology.orchestra.run.smithi003.stdout:checking liburing.h usability... no 2023-11-04T05:30:17.446 INFO:teuthology.orchestra.run.smithi003.stdout:checking liburing.h presence... no 2023-11-04T05:30:17.447 INFO:teuthology.orchestra.run.smithi003.stdout:checking for liburing.h... no 2023-11-04T05:30:17.468 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/capability.h usability... no 2023-11-04T05:30:17.477 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/capability.h presence... no 2023-11-04T05:30:17.478 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/capability.h... no 2023-11-04T05:30:17.500 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/fiemap.h usability... yes 2023-11-04T05:30:17.507 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/fiemap.h presence... yes 2023-11-04T05:30:17.507 INFO:teuthology.orchestra.run.smithi003.stdout:checking for linux/fiemap.h... yes 2023-11-04T05:30:17.540 INFO:teuthology.orchestra.run.smithi003.stdout:checking for fallocate... yes 2023-11-04T05:30:17.573 INFO:teuthology.orchestra.run.smithi003.stdout:checking for open_by_handle_at... yes 2023-11-04T05:30:17.597 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/prctl.h usability... yes 2023-11-04T05:30:17.604 INFO:teuthology.orchestra.run.smithi003.stdout:checking sys/prctl.h presence... yes 2023-11-04T05:30:17.604 INFO:teuthology.orchestra.run.smithi003.stdout:checking for sys/prctl.h... yes 2023-11-04T05:30:17.629 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/fs.h usability... yes 2023-11-04T05:30:17.636 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/fs.h presence... yes 2023-11-04T05:30:17.636 INFO:teuthology.orchestra.run.smithi003.stdout:checking for linux/fs.h... yes 2023-11-04T05:30:17.657 INFO:teuthology.orchestra.run.smithi003.stdout:checking btrfsutil.h usability... no 2023-11-04T05:30:17.667 INFO:teuthology.orchestra.run.smithi003.stdout:checking btrfsutil.h presence... no 2023-11-04T05:30:17.667 INFO:teuthology.orchestra.run.smithi003.stdout:checking for btrfsutil.h... no 2023-11-04T05:30:17.731 INFO:teuthology.orchestra.run.smithi003.stdout:checking for copy_file_range... yes 2023-11-04T05:30:17.766 INFO:teuthology.orchestra.run.smithi003.stdout:checking for SEEK_DATA... yes 2023-11-04T05:30:17.813 INFO:teuthology.orchestra.run.smithi003.stdout:checking for BMV_OF_SHARED... yes 2023-11-04T05:30:17.855 INFO:teuthology.orchestra.run.smithi003.stdout:checking for nftw... yes 2023-11-04T05:30:17.889 INFO:teuthology.orchestra.run.smithi003.stdout:checking for RLIMIT_NOFILE... yes 2023-11-04T05:30:17.906 INFO:teuthology.orchestra.run.smithi003.stdout:checking for FIEXCHANGE_RANGE... no 2023-11-04T05:30:17.939 INFO:teuthology.orchestra.run.smithi003.stdout:checking for renameat2... yes 2023-11-04T05:30:17.972 INFO:teuthology.orchestra.run.smithi003.stdout:checking for reallocarray... yes 2023-11-04T05:30:17.985 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct mount_attr... no 2023-11-04T05:30:18.028 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_qgroup_limit... yes 2023-11-04T05:30:18.063 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-11-04T05:30:18.097 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-11-04T05:30:18.132 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-11-04T05:30:18.166 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-11-04T05:30:18.201 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-11-04T05:30:18.236 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-11-04T05:30:18.263 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/btrfs.h usability... yes 2023-11-04T05:30:18.270 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/btrfs.h presence... yes 2023-11-04T05:30:18.271 INFO:teuthology.orchestra.run.smithi003.stdout:checking for linux/btrfs.h... yes 2023-11-04T05:30:18.300 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/btrfs_tree.h usability... yes 2023-11-04T05:30:18.308 INFO:teuthology.orchestra.run.smithi003.stdout:checking linux/btrfs_tree.h presence... yes 2023-11-04T05:30:18.308 INFO:teuthology.orchestra.run.smithi003.stdout:checking for linux/btrfs_tree.h... yes 2023-11-04T05:30:18.339 INFO:teuthology.orchestra.run.smithi003.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-11-04T05:30:18.355 INFO:teuthology.orchestra.run.smithi003.stdout:configure: creating ./config.status 2023-11-04T05:30:18.659 INFO:teuthology.orchestra.run.smithi003.stdout:config.status: creating include/builddefs 2023-11-04T05:30:18.670 INFO:teuthology.orchestra.run.smithi003.stdout:config.status: creating include/config.h 2023-11-04T05:30:18.678 INFO:teuthology.orchestra.run.smithi003.stdout:config.status: executing libtool commands 2023-11-04T05:30:18.716 INFO:teuthology.orchestra.run.smithi003.stdout:Building include 2023-11-04T05:30:18.719 INFO:teuthology.orchestra.run.smithi003.stdout:Building lib 2023-11-04T05:30:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: [LTDEP] 2023-11-04T05:30:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dataascii.lo 2023-11-04T05:30:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] databin.lo 2023-11-04T05:30:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] datapid.lo 2023-11-04T05:30:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] file_lock.lo 2023-11-04T05:30:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] forker.lo 2023-11-04T05:30:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] pattern.lo 2023-11-04T05:30:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] open_flags.lo 2023-11-04T05:30:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] random_range.lo 2023-11-04T05:30:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] string_to_tokens.lo 2023-11-04T05:30:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] str_to_bytes.lo 2023-11-04T05:30:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] tlibio.lo 2023-11-04T05:30:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] write_log.lo 2023-11-04T05:30:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] random.lo 2023-11-04T05:30:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: [LD] 2023-11-04T05:30:20.772 INFO:teuthology.orchestra.run.smithi003.stdout:Building ltp 2023-11-04T05:30:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: [DEP] 2023-11-04T05:30:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] doio 2023-11-04T05:30:21.527 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fsstress 2023-11-04T05:30:21.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:30:21.695 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:30:22.786 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fsx 2023-11-04T05:30:23.607 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] growfiles 2023-11-04T05:30:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] iogen 2023-11-04T05:30:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-stress 2023-11-04T05:30:24.808 INFO:teuthology.orchestra.run.smithi003.stdout:Building src 2023-11-04T05:30:24.813 INFO:teuthology.orchestra.run.smithi003.stdout: [DEP] 2023-11-04T05:30:26.244 INFO:teuthology.orchestra.run.smithi003.stdout: [DEP] 2023-11-04T05:30:26.270 INFO:teuthology.orchestra.run.smithi003.stdout: [DEP] 2023-11-04T05:30:26.277 INFO:teuthology.orchestra.run.smithi003.stdout: [DEP] 2023-11-04T05:30:26.466 INFO:teuthology.orchestra.run.smithi003.stdout: [DEP] 2023-11-04T05:30:26.487 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dirstress 2023-11-04T05:30:26.707 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fill 2023-11-04T05:30:26.851 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fill2 2023-11-04T05:30:27.063 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] getpagesize 2023-11-04T05:30:27.192 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] holes 2023-11-04T05:30:27.380 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] lstat64 2023-11-04T05:30:27.550 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] nametest 2023-11-04T05:30:27.772 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] permname 2023-11-04T05:30:27.959 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] randholes 2023-11-04T05:30:28.191 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] runas 2023-11-04T05:30:28.354 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] truncfile 2023-11-04T05:30:28.522 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] usemem 2023-11-04T05:30:28.663 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] mmapcat 2023-11-04T05:30:28.803 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] append_reader 2023-11-04T05:30:28.945 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] append_writer 2023-11-04T05:30:29.087 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dirperf 2023-11-04T05:30:29.292 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] metaperf 2023-11-04T05:30:29.535 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] devzero 2023-11-04T05:30:29.699 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] feature 2023-11-04T05:30:29.900 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] alloc 2023-11-04T05:30:30.098 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fault 2023-11-04T05:30:30.252 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fstest 2023-11-04T05:30:30.473 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_access_root 2023-11-04T05:30:30.617 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] godown 2023-11-04T05:30:30.775 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] resvtest 2023-11-04T05:30:30.950 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] writemod 2023-11-04T05:30:31.086 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] writev_on_pagefault 2023-11-04T05:30:31.236 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] makeextents 2023-11-04T05:30:31.403 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] itrash 2023-11-04T05:30:31.550 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] rename 2023-11-04T05:30:31.686 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] multi_open_unlink 2023-11-04T05:30:31.839 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] unwritten_sync 2023-11-04T05:30:32.008 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] genhashnames 2023-11-04T05:30:32.183 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_holes 2023-11-04T05:30:32.322 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_writev 2023-11-04T05:30:32.462 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_truncate_cmtime 2023-11-04T05:30:32.611 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dirhash_collide 2023-11-04T05:30:32.780 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_rename_overwrite 2023-11-04T05:30:32.914 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] holetest 2023-11-04T05:30:33.119 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_truncate_self 2023-11-04T05:30:33.251 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] af_unix 2023-11-04T05:30:33.395 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_stale_pmd 2023-11-04T05:30:33.538 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_cow_race 2023-11-04T05:30:33.693 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_fallocate 2023-11-04T05:30:33.836 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fsync-err 2023-11-04T05:30:34.010 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_write_ro 2023-11-04T05:30:34.152 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_ext4_dax_journal_corruption 2023-11-04T05:30:34.301 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_ext4_dax_inline_corruption 2023-11-04T05:30:34.447 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_ofd_locks 2023-11-04T05:30:34.631 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_collision 2023-11-04T05:30:34.816 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] mmap-write-concurrent 2023-11-04T05:30:34.976 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_get_file_time 2023-11-04T05:30:35.115 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_create_short_dirs 2023-11-04T05:30:35.272 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_create_long_dirs 2023-11-04T05:30:35.426 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_enospc 2023-11-04T05:30:35.600 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_writev_overlap 2023-11-04T05:30:35.758 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] checkpoint_journal 2023-11-04T05:30:35.903 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] mmap-rw-fault 2023-11-04T05:30:36.076 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] allocstale 2023-11-04T05:30:36.234 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_cow_memory_failure 2023-11-04T05:30:36.394 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fake-dump-rootino 2023-11-04T05:30:36.535 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dio-buf-fault 2023-11-04T05:30:36.680 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] rewinddir-test 2023-11-04T05:30:36.838 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] xfsctl 2023-11-04T05:30:37.014 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] bstat 2023-11-04T05:30:37.204 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mtab 2023-11-04T05:30:37.377 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] getdevicesize 2023-11-04T05:30:37.512 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] preallo_rw_pattern_reader 2023-11-04T05:30:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] preallo_rw_pattern_writer 2023-11-04T05:30:37.903 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] ftrunc 2023-11-04T05:30:38.043 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] trunc 2023-11-04T05:30:38.195 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fs_perms 2023-11-04T05:30:38.351 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] testx 2023-11-04T05:30:38.477 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] looptest 2023-11-04T05:30:38.657 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] locktest 2023-11-04T05:30:38.946 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] unwritten_mmap 2023-11-04T05:30:39.099 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] bulkstat_unlink_test 2023-11-04T05:30:39.287 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] deduperace 2023-11-04T05:30:39.469 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] bulkstat_unlink_test_modified 2023-11-04T05:30:39.652 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_dir_offset 2023-11-04T05:30:39.792 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_futimens 2023-11-04T05:30:39.927 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_immutable 2023-11-04T05:30:40.495 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] stale_handle 2023-11-04T05:30:40.662 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] pwrite_mmap_blocked 2023-11-04T05:30:40.806 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_dir_offset2 2023-11-04T05:30:40.985 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] seek_sanity_test 2023-11-04T05:30:41.375 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] seek_copy_test 2023-11-04T05:30:41.540 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_readdir_1 2023-11-04T05:30:41.679 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_readdir_2 2023-11-04T05:30:41.823 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fsync-tester 2023-11-04T05:30:42.056 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] nsexec 2023-11-04T05:30:42.222 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] cloner 2023-11-04T05:30:42.389 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] renameat2 2023-11-04T05:30:42.545 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_getcwd 2023-11-04T05:30:42.693 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] e4compact 2023-11-04T05:30:42.881 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] test-nextquota 2023-11-04T05:30:43.041 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] punch-alternating 2023-11-04T05:30:43.201 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] attr-list-by-handle-cursor-test 2023-11-04T05:30:43.360 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] listxattr 2023-11-04T05:30:43.499 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dio-interleaved 2023-11-04T05:30:43.651 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_dir_type 2023-11-04T05:30:43.798 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] dio-invalidate-cache 2023-11-04T05:30:43.989 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] stat_test 2023-11-04T05:30:44.267 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_encrypted_d_revalidate 2023-11-04T05:30:44.423 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] attr_replace_test 2023-11-04T05:30:44.576 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] swapon 2023-11-04T05:30:44.732 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] mkswap 2023-11-04T05:30:44.875 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_attr_corruption 2023-11-04T05:30:45.017 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_open_tmpfiles 2023-11-04T05:30:45.186 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fscrypt-crypt-util 2023-11-04T05:30:45.783 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] bulkstat_null_ocount 2023-11-04T05:30:45.932 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] splice-test 2023-11-04T05:30:46.103 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] chprojid_fail 2023-11-04T05:30:46.256 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] detached_mounts_propagation 2023-11-04T05:30:46.429 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] ext4_resize 2023-11-04T05:30:46.569 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_readdir_3 2023-11-04T05:30:46.752 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] splice2pipe 2023-11-04T05:30:46.909 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] uuid_ioctl 2023-11-04T05:30:47.056 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fiemap-tester 2023-11-04T05:30:47.308 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_stripealign 2023-11-04T05:30:47.454 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] open_by_handle 2023-11-04T05:30:47.692 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] t_mmap_dio 2023-11-04T05:30:47.845 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] xfsfind 2023-11-04T05:30:48.026 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] fssum 2023-11-04T05:30:48.431 INFO:teuthology.orchestra.run.smithi003.stdout:Building log-writes 2023-11-04T05:30:48.437 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] replay-log 2023-11-04T05:30:48.726 INFO:teuthology.orchestra.run.smithi003.stdout:Building perf 2023-11-04T05:30:48.729 INFO:teuthology.orchestra.run.smithi003.stdout:Building aio-dio-regress 2023-11-04T05:30:48.739 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-append-write-fallocate-race 2023-11-04T05:30:48.900 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-fcntl-race 2023-11-04T05:30:49.056 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-subblock-eof-read 2023-11-04T05:30:49.196 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-append-write-read-race 2023-11-04T05:30:49.365 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-invalidate-readahead 2023-11-04T05:30:49.523 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-write-verify 2023-11-04T05:30:49.709 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-11-04T05:30:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-last-ref-held-by-io 2023-11-04T05:30:50.008 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-cycle-write 2023-11-04T05:30:50.185 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-extend-stat 2023-11-04T05:30:50.344 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-11-04T05:30:50.473 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-hole-filling-race 2023-11-04T05:30:50.617 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-invalidate-failure 2023-11-04T05:30:50.770 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aiodio_sparse2 2023-11-04T05:30:50.976 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aiocp 2023-11-04T05:30:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-cow-race 2023-11-04T05:30:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] aio-dio-eof-race 2023-11-04T05:30:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:Building vfs 2023-11-04T05:30:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] vfstest 2023-11-04T05:30:51.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:30:51.846 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:30:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: [CC] mount-idmapped 2023-11-04T05:30:53.608 INFO:teuthology.orchestra.run.smithi003.stdout:Building m4 2023-11-04T05:30:53.611 INFO:teuthology.orchestra.run.smithi003.stdout:Building common 2023-11-04T05:30:53.615 INFO:teuthology.orchestra.run.smithi003.stdout:Building tests 2023-11-04T05:30:53.620 INFO:teuthology.orchestra.run.smithi003.stdout:Building perf 2023-11-04T05:30:53.625 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/perf/group.list 2023-11-04T05:30:53.635 INFO:teuthology.orchestra.run.smithi003.stdout:Building ext4 2023-11-04T05:30:53.641 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ext4/group.list 2023-11-04T05:30:53.675 INFO:teuthology.orchestra.run.smithi003.stdout:Building ocfs2 2023-11-04T05:30:53.680 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ocfs2/group.list 2023-11-04T05:30:53.692 INFO:teuthology.orchestra.run.smithi003.stdout:Building overlay 2023-11-04T05:30:53.697 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/overlay/group.list 2023-11-04T05:30:53.734 INFO:teuthology.orchestra.run.smithi003.stdout:Building cifs 2023-11-04T05:30:53.739 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/cifs/group.list 2023-11-04T05:30:53.751 INFO:teuthology.orchestra.run.smithi003.stdout:Building generic 2023-11-04T05:30:53.757 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/generic/group.list 2023-11-04T05:30:53.851 INFO:teuthology.orchestra.run.smithi003.stdout:Building shared 2023-11-04T05:30:53.856 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/shared/group.list 2023-11-04T05:30:53.871 INFO:teuthology.orchestra.run.smithi003.stdout:Building nfs 2023-11-04T05:30:53.876 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/nfs/group.list 2023-11-04T05:30:53.889 INFO:teuthology.orchestra.run.smithi003.stdout:Building f2fs 2023-11-04T05:30:53.894 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/f2fs/group.list 2023-11-04T05:30:53.910 INFO:teuthology.orchestra.run.smithi003.stdout:Building udf 2023-11-04T05:30:53.915 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/udf/group.list 2023-11-04T05:30:53.925 INFO:teuthology.orchestra.run.smithi003.stdout:Building ceph 2023-11-04T05:30:53.930 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ceph/group.list 2023-11-04T05:30:53.944 INFO:teuthology.orchestra.run.smithi003.stdout:Building xfs 2023-11-04T05:30:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/xfs/group.list 2023-11-04T05:30:54.044 INFO:teuthology.orchestra.run.smithi003.stdout:Building btrfs 2023-11-04T05:30:54.050 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/btrfs/group.list 2023-11-04T05:30:54.106 INFO:teuthology.orchestra.run.smithi003.stdout:Building tmpfs 2023-11-04T05:30:54.111 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/tmpfs/group.list 2023-11-04T05:30:54.123 INFO:teuthology.orchestra.run.smithi003.stdout:Building selftest 2023-11-04T05:30:54.128 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/selftest/group.list 2023-11-04T05:30:54.141 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo make install) 2023-11-04T05:30:54.178 INFO:teuthology.orchestra.run.smithi003.stdout:Building include 2023-11-04T05:30:54.181 INFO:teuthology.orchestra.run.smithi003.stdout:Building lib 2023-11-04T05:30:54.185 INFO:teuthology.orchestra.run.smithi003.stdout:Building ltp 2023-11-04T05:30:54.190 INFO:teuthology.orchestra.run.smithi003.stdout:Building src 2023-11-04T05:30:54.222 INFO:teuthology.orchestra.run.smithi003.stdout:Building log-writes 2023-11-04T05:30:54.226 INFO:teuthology.orchestra.run.smithi003.stdout:Building perf 2023-11-04T05:30:54.229 INFO:teuthology.orchestra.run.smithi003.stdout:Building aio-dio-regress 2023-11-04T05:30:54.233 INFO:teuthology.orchestra.run.smithi003.stdout:Building vfs 2023-11-04T05:30:54.237 INFO:teuthology.orchestra.run.smithi003.stdout:Building m4 2023-11-04T05:30:54.240 INFO:teuthology.orchestra.run.smithi003.stdout:Building common 2023-11-04T05:30:54.243 INFO:teuthology.orchestra.run.smithi003.stdout:Building tests 2023-11-04T05:30:54.249 INFO:teuthology.orchestra.run.smithi003.stdout:Building perf 2023-11-04T05:30:54.254 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/perf/group.list 2023-11-04T05:30:54.265 INFO:teuthology.orchestra.run.smithi003.stdout:Building ext4 2023-11-04T05:30:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ext4/group.list 2023-11-04T05:30:54.304 INFO:teuthology.orchestra.run.smithi003.stdout:Building ocfs2 2023-11-04T05:30:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ocfs2/group.list 2023-11-04T05:30:54.321 INFO:teuthology.orchestra.run.smithi003.stdout:Building overlay 2023-11-04T05:30:54.326 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/overlay/group.list 2023-11-04T05:30:54.363 INFO:teuthology.orchestra.run.smithi003.stdout:Building cifs 2023-11-04T05:30:54.368 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/cifs/group.list 2023-11-04T05:30:54.379 INFO:teuthology.orchestra.run.smithi003.stdout:Building generic 2023-11-04T05:30:54.386 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/generic/group.list 2023-11-04T05:30:54.478 INFO:teuthology.orchestra.run.smithi003.stdout:Building shared 2023-11-04T05:30:54.482 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/shared/group.list 2023-11-04T05:30:54.498 INFO:teuthology.orchestra.run.smithi003.stdout:Building nfs 2023-11-04T05:30:54.503 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/nfs/group.list 2023-11-04T05:30:54.516 INFO:teuthology.orchestra.run.smithi003.stdout:Building f2fs 2023-11-04T05:30:54.521 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/f2fs/group.list 2023-11-04T05:30:54.537 INFO:teuthology.orchestra.run.smithi003.stdout:Building udf 2023-11-04T05:30:54.542 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/udf/group.list 2023-11-04T05:30:54.552 INFO:teuthology.orchestra.run.smithi003.stdout:Building ceph 2023-11-04T05:30:54.558 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ceph/group.list 2023-11-04T05:30:54.570 INFO:teuthology.orchestra.run.smithi003.stdout:Building xfs 2023-11-04T05:30:54.577 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/xfs/group.list 2023-11-04T05:30:54.668 INFO:teuthology.orchestra.run.smithi003.stdout:Building btrfs 2023-11-04T05:30:54.674 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/btrfs/group.list 2023-11-04T05:30:54.729 INFO:teuthology.orchestra.run.smithi003.stdout:Building tmpfs 2023-11-04T05:30:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/tmpfs/group.list 2023-11-04T05:30:54.746 INFO:teuthology.orchestra.run.smithi003.stdout:Building selftest 2023-11-04T05:30:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: [GROUP] /tmp/tmp.ByRml5cCmzxfstests-dev/tests/selftest/group.list 2023-11-04T05:30:54.762 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-11-04T05:30:54.764 INFO:teuthology.orchestra.run.smithi003.stdout:gmake[1]: Nothing to be done for 'install'. 2023-11-04T05:30:54.764 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-11-04T05:30:54.766 INFO:teuthology.orchestra.run.smithi003.stdout:gmake[1]: Nothing to be done for 'install'. 2023-11-04T05:30:54.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-11-04T05:30:54.768 INFO:teuthology.orchestra.run.smithi003.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-11-04T05:30:54.777 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-11-04T05:30:54.857 INFO:teuthology.orchestra.run.smithi003.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-11-04T05:30:54.866 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-11-04T05:30:54.882 INFO:teuthology.orchestra.run.smithi003.stdout:Building log-writes 2023-11-04T05:30:54.886 INFO:teuthology.orchestra.run.smithi003.stdout:Building perf 2023-11-04T05:30:54.889 INFO:teuthology.orchestra.run.smithi003.stdout:Building aio-dio-regress 2023-11-04T05:30:54.893 INFO:teuthology.orchestra.run.smithi003.stdout:Building vfs 2023-11-04T05:30:54.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake -C log-writes install 2023-11-04T05:30:54.897 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-11-04T05:30:54.906 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-11-04T05:30:54.915 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake -C perf install 2023-11-04T05:30:54.917 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-11-04T05:30:54.926 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-11-04T05:30:54.949 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-11-04T05:30:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-11-04T05:30:54.960 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 aio-dio-append-write-fallocate-race aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-hole-filling-race aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress 2023-11-04T05:30:55.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake -C vfs install 2023-11-04T05:30:55.016 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-11-04T05:30:55.025 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-11-04T05:30:55.037 INFO:teuthology.orchestra.run.smithi003.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-11-04T05:30:55.046 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-11-04T05:30:56.278 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2023-11-04T05:30:56.383 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-11-04T05:30:56.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-11-04T05:30:56.416 INFO:teuthology.orchestra.run.smithi003.stdout:gmake[1]: Nothing to be done for 'install'. 2023-11-04T05:30:56.416 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-11-04T05:30:56.418 INFO:teuthology.orchestra.run.smithi003.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-11-04T05:30:56.426 INFO:teuthology.orchestra.run.smithi003.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-11-04T05:30:56.567 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-11-04T05:30:56.569 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/btrfs/ install 2023-11-04T05:30:56.571 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-11-04T05:30:56.580 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 /var/lib/xfstests/tests/btrfs 2023-11-04T05:30:57.431 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-11-04T05:30:57.440 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 240.out 092.out 077.out 140.out 265.out 236.out 108.out 121.out 204.out 233.out 194.out 055.out 295.out 109.out 033.out 082.out 252.out 270.out 040.out 176.out 202.out 137.out 102.out 034.out 125.out 014.out 165.out 181.out 152.out 024.out 243.out 192.out 148.out 282.out 267.out 178.out 052.out 256.out 228.out 183.out 273.out 251.out 043.out 069.out 027.out 151.out 206.out 195.out 274.out 215.out 115.out 275.out 250.out 191.out 281.out 046.out 051.out 131.out 244.out 219.out 119.out 019.out 128.out 232.out 020.out 263.out 171.out 261.out 045.out 217.out 117.out 017.out 072.out 068.out 048.out 062.out 038.out 084.out 129.out 066.out 029.out 229.out 103.out 111.out 011.out 234.out 134.out 170.out 260.out 157.out 143.out 133.out 286.out 161.out 246.out 056.out 216.out 116.out 016.out 155.out 296.out 118.out 287.out 067.out 010.out 238.out 141.out 276.out 114.out 138.out 298.out 095.out 297.out 160.out 091.out 081.out 071.out 061.out 085.out 037.out 300.out 200.out 100.out 025.out 258.out 185.out 269.out 208.out 039.out 173.out 222.out 035.out 169.out 098.out 290.out 080.out 070.out 060.out 207.out 106.out 009.out 167.out 211.out 158.out 182.out 146.out 188.out 293.out 104.out 089.out 086.out 254.out 177.out 225.out 174.out 268.out 135.out 124.out 172.out 063.out 058.out 113.out 262.out 042.out 047.out 193.out 097.out 271.out 283.out 083.out 199.out 099.out 278.out 015.out 007.out 209.out 053.out 279.out 164.out 154.out 239.out 241.out 002.out 301.out 190.out 214.out 074.out 036.out 041.out 057.out 175.out 186.out 005.out 223.out 150.out 123.out 179.out 144.out 032.out 277.out 205.out 136.out 096.out 198.out 008.out 280.out 050.out 105.out 257.out 291.out 187.out 153.out 088.out 285.out 044.out 201.out 101.out 001.out 266.out 237.out 065.out 231.out 227.out 075.out 028.out 213.out 142.out 013.out 031.out 272.out 078.out 184.out 259.out 210.out 132.out 093.out 235.out 127.out 230.out 166.out 292.out 130.out 023.out 073.out 030.out 203.out 003.out 245.out 145.out 159.out 249.out 122.out 076.out 022.out 289.out 006.out 284.out 299.out 224.out 156.out 110.out 149.out 064.out 264.out 218.out 018.out 168.out 021.out 079.out 054.out 059.out 049.out 226.out 163.out 126.out 026.out 180.out 253.out 255.out 139.out 090.out 221.out 087.out 294.out 212.out 112.out 094.out 012.out 247.out 107.out 248.out 162.out 147.out 288.out 197.out 196.out 242.out 004.out 189.out 220.out 120.out /var/lib/xfstests/tests/btrfs 2023-11-04T05:30:58.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ceph/ install 2023-11-04T05:30:58.292 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-11-04T05:30:58.300 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-11-04T05:30:58.320 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-11-04T05:30:58.329 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-11-04T05:30:58.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/cifs/ install 2023-11-04T05:30:58.351 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-11-04T05:30:58.360 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-11-04T05:30:58.368 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-11-04T05:30:58.377 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-11-04T05:30:58.386 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ext4/ install 2023-11-04T05:30:58.388 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-11-04T05:30:58.397 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-11-04T05:30:58.592 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-11-04T05:30:58.601 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 032.out 039.out 305.out 005.out 008.out 035.out 036.out 308.out 050.out 040.out 306.out 007.out 021.out 046.out 012.out 009.out 033.out 045.out 034.out 056.out 047.out 018.out 023.out 302.out 020.out 002.out 014.out 024.out 013.out 031.out 015.out 304.out 004.out 048.out 038.out 052.out 303.out 029.out 011.out 027.out 042.out 044.out 055.out 030.out 058.out 003.out 017.out 043.out 053.out 016.out 059.out 037.out 301.out 010.out 022.out 006.out 057.out 025.out 026.out 271.out 051.out 041.out 307.out 019.out 028.out 054.out 049.out /var/lib/xfstests/tests/ext4 2023-11-04T05:30:58.807 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/f2fs/ install 2023-11-04T05:30:58.808 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-11-04T05:30:58.817 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-11-04T05:30:58.828 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-11-04T05:30:58.837 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-11-04T05:30:58.849 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/generic/ install 2023-11-04T05:30:58.853 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-11-04T05:30:58.862 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 /var/lib/xfstests/tests/generic 2023-11-04T05:31:00.934 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-11-04T05:31:00.943 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 050.out.xfsquota 035.out.nfs 035.out.default 050.out.nojournal 050.out.default 050.cfg 035.cfg 670.out 040.out 104.out 575.out 537.out 683.out 497.out 337.out 702.out 460.out 149.out 088.out 366.out 202.out 137.out 034.out 110.out 389.out 403.out 593.out 571.out 243.out 592.out 282.out 472.out 468.out 628.out 528.out 607.out 428.out 659.out 407.out 228.out 128.out 171.out 028.out 620.out 273.out 220.out 474.out 492.out 609.out 707.out 693.out 595.out 506.out 732.out 268.out 632.out 532.out 591.out 281.out 471.out 619.out 519.out 013.out 419.out 319.out 177.out 523.out 119.out 019.out 727.out 372.out 718.out 143.out 716.out 161.out 516.out 518.out 538.out 655.out 216.out 116.out 155.out 118.out 338.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 457.out 333.out 258.out 362.out 379.out 691.out 039.out 173.out 498.out 665.out 098.out 426.out 290.out 573.out 706.out 060.out 406.out 306.out 106.out 642.out 447.out 217.out 120.out 191.out 411.out 357.out 481.out 211.out 485.out 014.out 182.out 370.out 188.out 030.out 099.out 585.out 724.out 721.out 624.out 524.out 174.out 156.out 324.out 224.out 313.out 355.out 124.out 382.out 672.out 278.out 063.out 058.out 384.out 296.out 042.out 641.out 271.out 465.out 473.out 335.out 247.out 509.out 568.out 441.out 440.out 475.out 548.out 356.out 341.out 094.out 241.out 600.out 612.out 714.out 614.out 514.out 414.out 314.out 416.out 214.out 381.out 056.out 041.out 016.out 340.out 358.out 572.out 586.out 521.out 240.out 549.out 095.out 633.out 434.out 140.out 708.out 533.out 508.out 408.out 308.out 208.out 433.out 087.out 199.out 458.out 233.out 639.out 443.out 540.out 053.out 279.out 276.out 711.out 164.out 376.out 589.out 587.out 339.out 294.out 352.out 484.out 489.out 139.out 317.out 399.out 552.out 695.out 017.out 677.out 097.out 175.out 315.out 295.out 580.out 193.out 186.out 452.out 679.out 292.out 252.out 077.out 167.out 688.out 206.out 348.out 394.out 651.out 466.out 697.out 495.out 725.out 109.out 482.out 625.out 283.out 525.out 488.out 496.out 325.out 089.out 225.out 322.out 636.out 125.out 715.out 192.out 544.out 515.out 650.out 152.out 658.out 563.out 393.out 578.out 551.out 015.out 392.out 682.out 684.out 550.out 052.out 652.out 534.out 351.out 583.out 451.out 450.out 463.out 251.out 267.out 176.out 069.out 644.out 065.out 487.out 615.out 395.out 676.out 415.out 344.out 215.out 115.out 444.out 027.out 250.out 391.out 674.out 464.out 705.out 536.out 577.out 439.out 244.out 336.out 144.out 505.out 405.out 057.out 205.out 136.out 598.out 363.out 036.out 198.out 723.out 008.out 390.out 680.out 605.out 305.out 105.out 005.out 367.out 291.out 719.out 323.out 685.out 562.out 570.out 388.out 527.out 158.out 237.out 151.out 227.out 163.out 332.out 661.out 232.out 478.out 584.out 462.out 597.out 412.out 132.out 179.out 037.out 293.out 499.out 638.out 448.out 353.out 020.out 660.out 561.out 141.out 076.out 668.out 044.out 648.out 387.out 594.out 438.out 265.out 289.out 574.out 006.out 690.out 284.out 567.out 299.out 456.out 461.out 477.out 307.out 647.out 493.out 107.out 007.out 380.out 526.out 479.out 547.out 150.out 326.out 082.out 480.out 226.out 413.out 126.out 025.out 121.out 026.out 021.out 694.out 347.out 275.out 090.out 361.out 564.out 302.out 432.out 002.out 558.out 162.out 147.out 288.out 197.out 033.out 396.out 051.out 350.out 145.out 047.out 024.out 360.out 543.out 261.out 045.out 692.out 255.out 726.out 729.out 378.out 346.out 048.out 062.out 626.out 529.out 231.out 343.out 662.out 329.out 165.out 334.out 103.out 003.out 129.out 423.out 371.out 234.out 183.out 671.out 645.out 673.out 190.out 686.out 312.out 486.out 223.out 123.out 262.out 669.out 023.out 459.out 616.out 032.out 266.out 146.out 398.out 728.out 696.out 427.out 590.out 280.out 470.out 455.out 187.out 091.out 046.out 731.out 374.out 631.out 256.out 328.out 260.out 654.out 359.out 531.out 649.out 301.out 101.out 635.out 623.out 385.out 331.out 556.out 713.out 613.out 513.out 622.out 131.out 213.out 730.out 634.out 031.out 565.out 630.out 675.out 272.out 530.out 510.out 627.out 184.out 576.out 430.out 310.out 569.out 454.out 259.out 210.out 449.out 327.out 330.out 067.out 093.out 604.out 287.out 127.out 699.out 166.out 130.out 219.out 073.out 603.out 503.out 303.out 203.out 354.out 157.out 386.out 349.out 055.out 245.out 709.out 522.out 422.out 222.out 159.out 539.out 249.out 122.out 689.out 022.out 681.out 424.out 321.out 066.out 490.out 435.out 375.out 134.out 154.out 566.out 618.out 501.out 418.out 318.out 542.out 218.out 018.out 377.out 172.out 168.out 079.out 469.out 054.out 059.out 049.out 442.out 657.out 621.out 180.out 270.out 253.out 421.out 712.out 667.out 512.out 494.out 221.out 212.out 112.out 342.out 012.out 029.out 195.out 554.out 397.out 185.out 596.out 720.out 248.out 196.out 520.out 242.out 420.out 285.out 320.out 629.out 230.out 602.out 502.out 402.out 102.out 717.out 617.out 517.out 429.out 417.out 365.out 092.out 117.out 664.out 072.out 068.out 229.out 038.out 084.out 369.out 611.out 511.out 311.out 368.out 637.out 111.out 011.out 257.out 194.out 170.out 606.out 133.out 286.out 043.out 437.out 545.out 559.out 710.out 610.out 687.out 410.out 701.out 601.out 401.out 201.out 010.out 001.out 096.out 114.out 113.out 074.out 581.out 491.out 081.out 148.out 071.out 083.out 653.out 700.out 064.out 400.out 300.out 200.out 100.out 646.out 541.out 445.out 269.out 546.out 656.out 698.out 553.out 178.out 298.out 446.out 080.out 070.out 608.out 507.out 409.out 309.out 209.out 108.out 009.out 643.out 004.out 557.out 453.out 666.out 236.out 078.out 364.out 555.out 075.out 582.out 383.out 085.out 704.out 588.out 504.out 404.out 304.out 204.out 535.out 425.out 345.out 246.out 467.out 086.out 254.out 207.out 142.out 235.out 436.out 703.out 135.out 264.out 431.out 678.out 373.out 316.out 169.out 640.out 579.out 297.out 153.out 560.out 663.out 599.out 476.out /var/lib/xfstests/tests/generic 2023-11-04T05:31:03.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/nfs/ install 2023-11-04T05:31:03.028 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-11-04T05:31:03.037 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-11-04T05:31:03.046 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-11-04T05:31:03.055 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-11-04T05:31:03.064 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/ocfs2/ install 2023-11-04T05:31:03.066 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-11-04T05:31:03.075 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-11-04T05:31:03.084 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-11-04T05:31:03.093 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-11-04T05:31:03.102 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/overlay/ install 2023-11-04T05:31:03.104 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-11-04T05:31:03.113 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-11-04T05:31:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-11-04T05:31:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 003.out 018.out 024.out 077.out 069.out 020.out 102.out 014.out 023.out 013.out 031.out 015.out 117.out 082.out 072.out 068.out 063.out 048.out 062.out 038.out 052.out 042.out 075.out 110.out 067.out 029.out 111.out 055.out 112.out 066.out 073.out 078.out 030.out 058.out 104.out 017.out 002.out 043.out 053.out 036.out 116.out 016.out 101.out 010.out 001.out 022.out 074.out 004.out 103.out 115.out 057.out 113.out 114.out 081.out 026.out 071.out 061.out 025.out 051.out 011.out 041.out 047.out 100.out 064.out 007.out 028.out 076.out 079.out 054.out 059.out 008.out 049.out 032.out 039.out 046.out 021.out 105.out 005.out 107.out 065.out 108.out 080.out 070.out 060.out 050.out 027.out 106.out 009.out 006.out 037.out 044.out 012.out 109.out 033.out 045.out 040.out 056.out 035.out 019.out 034.out /var/lib/xfstests/tests/overlay 2023-11-04T05:31:03.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/perf/ install 2023-11-04T05:31:03.695 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-11-04T05:31:03.704 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-11-04T05:31:03.713 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-11-04T05:31:03.722 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-11-04T05:31:03.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/selftest/ install 2023-11-04T05:31:03.732 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-11-04T05:31:03.741 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-11-04T05:31:03.764 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-11-04T05:31:03.773 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 006.out 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-11-04T05:31:03.796 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/shared/ install 2023-11-04T05:31:03.797 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-11-04T05:31:03.806 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-11-04T05:31:03.821 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-11-04T05:31:03.830 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-11-04T05:31:03.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/tmpfs/ install 2023-11-04T05:31:03.846 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-11-04T05:31:03.855 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-11-04T05:31:03.864 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-11-04T05:31:03.873 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-11-04T05:31:03.882 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/udf/ install 2023-11-04T05:31:03.884 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-11-04T05:31:03.893 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-11-04T05:31:03.901 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-11-04T05:31:03.910 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-11-04T05:31:03.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ByRml5cCmzxfstests-dev/tests/xfs/ install 2023-11-04T05:31:03.923 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-11-04T05:31:03.932 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-11-04T05:31:05.896 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-11-04T05:31:05.905 INFO:teuthology.orchestra.run.smithi003.stdout:../../install-sh -o root -g root -m 644 096.out.internal 033.out.crc 096.out.external 033.out.default 071.out.64 216.out.classic 216.out.64mblog 071.out.32 216.cfg 033.cfg 295.out 497.out 337.out 385.out 082.out 270.out 460.out 149.out 759.out 502.out 088.out 366.out 202.out 137.out 034.out 110.out 389.out 018.out 593.out 571.out 243.out 592.out 282.out 267.out 728.out 468.out 799.out 284.out 528.out 769.out 428.out 507.out 407.out 228.out 128.out 171.out 183.out 273.out 017.out 492.out 109.out 559.out 506.out 195.out 732.out 268.out 275.out 600.out 113.out 532.out 591.out 281.out 471.out 723.out 255.out 013.out 371.out 319.out 467.out 523.out 119.out 775.out 727.out 372.out 165.out 229.out 103.out 003.out 129.out 131.out 383.out 738.out 260.out 718.out 143.out 716.out 161.out 516.out 518.out 416.out 026.out 116.out 043.out 155.out 338.out 238.out 500.out 138.out 189.out 263.out 184.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 047.out 772.out 258.out 379.out 322.out 039.out 430.out 173.out 222.out 742.out 098.out 426.out 290.out 480.out 060.out 406.out 306.out 141.out 106.out 006.out 167.out 191.out 542.out 211.out 442.out 188.out 099.out 177.out 721.out 174.out 156.out 771.out 324.out 224.out 313.out 124.out 278.out 058.out 741.out 384.out 042.out 193.out 392.out 768.out 399.out 473.out 770.out 709.out 247.out 568.out 364.out 209.out 441.out 440.out 009.out 740.out 356.out 197.out 341.out 094.out 241.out 501.out 089.out 540.out 514.out 414.out 314.out 214.out 381.out 147.out 041.out 340.out 793.out 358.out 572.out 586.out 240.out 549.out 092.out 439.out 434.out 140.out 708.out 766.out 508.out 408.out 308.out 208.out 737.out 040.out 304.out 349.out 254.out 537.out 059.out 086.out 249.out 335.out 142.out 235.out 436.out 575.out 253.out 715.out 533.out 264.out 431.out 784.out 433.out 036.out 316.out 169.out 035.out 579.out 457.out 297.out 393.out 333.out 599.out 476.out 087.out 199.out 739.out 233.out 752.out 736.out 443.out 773.out 053.out 749.out 279.out 133.out 711.out 458.out 164.out 589.out 587.out 339.out 577.out 294.out 352.out 484.out 498.out 489.out 139.out 774.out 790.out 552.out 781.out 357.out 065.out 097.out 175.out 186.out 452.out 069.out 252.out 751.out 376.out 724.out 206.out 778.out 190.out 794.out 348.out 466.out 248.out 725.out 785.out 750.out 525.out 496.out 325.out 257.out 225.out 125.out 789.out 192.out 296.out 152.out 563.out 246.out 551.out 015.out 792.out 075.out 472.out 578.out 463.out 550.out 052.out 534.out 351.out 276.out 451.out 425.out 450.out 744.out 251.out 444.out 487.out 762.out 395.out 515.out 415.out 315.out 215.out 115.out 597.out 027.out 250.out 344.out 051.out 536.out 030.out 244.out 336.out 028.out 150.out 144.out 505.out 342.out 136.out 598.out 005.out 176.out 198.out 008.out 390.out 178.out 405.out 305.out 205.out 105.out 367.out 291.out 437.out 044.out 719.out 323.out 519.out 363.out 562.out 382.out 570.out 388.out 527.out 158.out 019.out 237.out 151.out 760.out 227.out 163.out 332.out 796.out 582.out 232.out 584.out 462.out 220.out 412.out 132.out 179.out 037.out 293.out 499.out 112.out 748.out 020.out 786.out 063.out 561.out 076.out 465.out 014.out 758.out 387.out 594.out 438.out 265.out 289.out 574.out 755.out 548.out 299.out 419.out 521.out 581.out 595.out 461.out 477.out 307.out 726.out 107.out 007.out 448.out 262.out 380.out 093.out 479.out 547.out 326.out 226.out 747.out 126.out 025.out 121.out 447.out 761.out 580.out 021.out 485.out 347.out 077.out 090.out 361.out 564.out 788.out 066.out 455.out 432.out 002.out 585.out 162.out 288.out 050.out 396.out 446.out 350.out 354.out 429.out 187.out 360.out 543.out 261.out 734.out 045.out 292.out 482.out 729.out 104.out 048.out 062.out 529.out 231.out 343.out 503.out 526.out 329.out 486.out 223.out 123.out 023.out 459.out 032.out 798.out 277.out 373.out 146.out 398.out 168.out 427.out 590.out 280.out 470.out 491.out 767.out 016.out 046.out 488.out 731.out 782.out 374.out 583.out 285.out 127.out 328.out 359.out 531.out 301.out 101.out 266.out 331.out 791.out 743.out 556.out 713.out 081.out 024.out 513.out 213.out 730.out 095.out 031.out 565.out 182.out 272.out 530.out 754.out 573.out 576.out 569.out 454.out 259.out 449.out 327.out 330.out 067.out 493.out 010.out 287.out 230.out 166.out 130.out 219.out 073.out 370.out 135.out 303.out 203.out 345.out 355.out 386.out 560.out 055.out 334.out 245.out 522.out 787.out 745.out 422.out 394.out 539.out 524.out 122.out 234.out 022.out 424.out 779.out 490.out 435.out 353.out 423.out 134.out 154.out 271.out 064.out 418.out 318.out 218.out 118.out 757.out 377.out 172.out 079.out 469.out 054.out 049.out 765.out 776.out 456.out 180.out 378.out 421.out 712.out 403.out 321.out 057.out 512.out 494.out 221.out 312.out 212.out 538.out 256.out 012.out 764.out 283.out 554.out 397.out 185.out 596.out 720.out 481.out 196.out 520.out 242.out 420.out 478.out 320.out 555.out 120.out 153.out 402.out 302.out 102.out 717.out 517.out 417.out 317.out 365.out 217.out 117.out 475.out 072.out 068.out 474.out 038.out 084.out 056.out 369.out 511.out 148.out 411.out 029.out 311.out 368.out 111.out 011.out 783.out 194.out 170.out 286.out 756.out 545.out 714.out 157.out 710.out 510.out 413.out 410.out 310.out 210.out 401.out 201.out 001.out 495.out 362.out 114.out 753.out 074.out 145.out 797.out 464.out 091.out 746.out 083.out 800.out 777.out 400.out 300.out 200.out 100.out 375.out 541.out 445.out 269.out 546.out 780.out 553.out 763.out 298.out 567.out 080.out 070.out 509.out 409.out 309.out 207.out 108.out 346.out 391.out 004.out 557.out 453.out 236.out 078.out 159.out 566.out 735.out 085.out 588.out 504.out 404.out 558.out 204.out 535.out 795.out 733.out 544.out /var/lib/xfstests/tests/xfs 2023-11-04T05:31:07.884 INFO:teuthology.orchestra.run.smithi003.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-11-04T05:31:07.892 INFO:teuthology.orchestra.run.smithi003.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-11-04T05:31:07.901 INFO:teuthology.orchestra.run.smithi003.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-11-04T05:31:07.910 INFO:teuthology.orchestra.run.smithi003.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-11-04T05:31:07.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T05:31:07.922 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec ls tests/generic) 2023-11-04T05:31:07.980 INFO:teuthology.orchestra.run.smithi003.stdout:001 2023-11-04T05:31:07.980 INFO:teuthology.orchestra.run.smithi003.stdout:001.out 2023-11-04T05:31:07.980 INFO:teuthology.orchestra.run.smithi003.stdout:002 2023-11-04T05:31:07.981 INFO:teuthology.orchestra.run.smithi003.stdout:002.out 2023-11-04T05:31:07.981 INFO:teuthology.orchestra.run.smithi003.stdout:003 2023-11-04T05:31:07.981 INFO:teuthology.orchestra.run.smithi003.stdout:003.out 2023-11-04T05:31:07.981 INFO:teuthology.orchestra.run.smithi003.stdout:004 2023-11-04T05:31:07.982 INFO:teuthology.orchestra.run.smithi003.stdout:004.out 2023-11-04T05:31:07.982 INFO:teuthology.orchestra.run.smithi003.stdout:005 2023-11-04T05:31:07.982 INFO:teuthology.orchestra.run.smithi003.stdout:005.out 2023-11-04T05:31:07.982 INFO:teuthology.orchestra.run.smithi003.stdout:006 2023-11-04T05:31:07.983 INFO:teuthology.orchestra.run.smithi003.stdout:006.out 2023-11-04T05:31:07.983 INFO:teuthology.orchestra.run.smithi003.stdout:007 2023-11-04T05:31:07.983 INFO:teuthology.orchestra.run.smithi003.stdout:007.out 2023-11-04T05:31:07.983 INFO:teuthology.orchestra.run.smithi003.stdout:008 2023-11-04T05:31:07.983 INFO:teuthology.orchestra.run.smithi003.stdout:008.out 2023-11-04T05:31:07.984 INFO:teuthology.orchestra.run.smithi003.stdout:009 2023-11-04T05:31:07.984 INFO:teuthology.orchestra.run.smithi003.stdout:009.out 2023-11-04T05:31:07.984 INFO:teuthology.orchestra.run.smithi003.stdout:010 2023-11-04T05:31:07.984 INFO:teuthology.orchestra.run.smithi003.stdout:010.out 2023-11-04T05:31:07.984 INFO:teuthology.orchestra.run.smithi003.stdout:011 2023-11-04T05:31:07.984 INFO:teuthology.orchestra.run.smithi003.stdout:011.out 2023-11-04T05:31:07.985 INFO:teuthology.orchestra.run.smithi003.stdout:012 2023-11-04T05:31:07.985 INFO:teuthology.orchestra.run.smithi003.stdout:012.out 2023-11-04T05:31:07.985 INFO:teuthology.orchestra.run.smithi003.stdout:013 2023-11-04T05:31:07.985 INFO:teuthology.orchestra.run.smithi003.stdout:013.out 2023-11-04T05:31:07.985 INFO:teuthology.orchestra.run.smithi003.stdout:014 2023-11-04T05:31:07.986 INFO:teuthology.orchestra.run.smithi003.stdout:014.out 2023-11-04T05:31:07.986 INFO:teuthology.orchestra.run.smithi003.stdout:015 2023-11-04T05:31:07.986 INFO:teuthology.orchestra.run.smithi003.stdout:015.out 2023-11-04T05:31:07.986 INFO:teuthology.orchestra.run.smithi003.stdout:016 2023-11-04T05:31:07.986 INFO:teuthology.orchestra.run.smithi003.stdout:016.out 2023-11-04T05:31:07.987 INFO:teuthology.orchestra.run.smithi003.stdout:017 2023-11-04T05:31:07.987 INFO:teuthology.orchestra.run.smithi003.stdout:017.out 2023-11-04T05:31:07.987 INFO:teuthology.orchestra.run.smithi003.stdout:018 2023-11-04T05:31:07.987 INFO:teuthology.orchestra.run.smithi003.stdout:018.out 2023-11-04T05:31:07.988 INFO:teuthology.orchestra.run.smithi003.stdout:019 2023-11-04T05:31:07.988 INFO:teuthology.orchestra.run.smithi003.stdout:019.out 2023-11-04T05:31:07.988 INFO:teuthology.orchestra.run.smithi003.stdout:020 2023-11-04T05:31:07.988 INFO:teuthology.orchestra.run.smithi003.stdout:020.out 2023-11-04T05:31:07.988 INFO:teuthology.orchestra.run.smithi003.stdout:021 2023-11-04T05:31:07.989 INFO:teuthology.orchestra.run.smithi003.stdout:021.out 2023-11-04T05:31:07.989 INFO:teuthology.orchestra.run.smithi003.stdout:022 2023-11-04T05:31:07.989 INFO:teuthology.orchestra.run.smithi003.stdout:022.out 2023-11-04T05:31:07.989 INFO:teuthology.orchestra.run.smithi003.stdout:023 2023-11-04T05:31:07.989 INFO:teuthology.orchestra.run.smithi003.stdout:023.out 2023-11-04T05:31:07.990 INFO:teuthology.orchestra.run.smithi003.stdout:024 2023-11-04T05:31:07.990 INFO:teuthology.orchestra.run.smithi003.stdout:024.out 2023-11-04T05:31:07.990 INFO:teuthology.orchestra.run.smithi003.stdout:025 2023-11-04T05:31:07.990 INFO:teuthology.orchestra.run.smithi003.stdout:025.out 2023-11-04T05:31:07.990 INFO:teuthology.orchestra.run.smithi003.stdout:026 2023-11-04T05:31:07.991 INFO:teuthology.orchestra.run.smithi003.stdout:026.out 2023-11-04T05:31:07.991 INFO:teuthology.orchestra.run.smithi003.stdout:027 2023-11-04T05:31:07.991 INFO:teuthology.orchestra.run.smithi003.stdout:027.out 2023-11-04T05:31:07.991 INFO:teuthology.orchestra.run.smithi003.stdout:028 2023-11-04T05:31:07.991 INFO:teuthology.orchestra.run.smithi003.stdout:028.out 2023-11-04T05:31:07.991 INFO:teuthology.orchestra.run.smithi003.stdout:029 2023-11-04T05:31:07.992 INFO:teuthology.orchestra.run.smithi003.stdout:029.out 2023-11-04T05:31:07.993 INFO:teuthology.orchestra.run.smithi003.stdout:030 2023-11-04T05:31:07.993 INFO:teuthology.orchestra.run.smithi003.stdout:030.out 2023-11-04T05:31:07.993 INFO:teuthology.orchestra.run.smithi003.stdout:031 2023-11-04T05:31:07.993 INFO:teuthology.orchestra.run.smithi003.stdout:031.out 2023-11-04T05:31:07.994 INFO:teuthology.orchestra.run.smithi003.stdout:032 2023-11-04T05:31:07.994 INFO:teuthology.orchestra.run.smithi003.stdout:032.out 2023-11-04T05:31:07.994 INFO:teuthology.orchestra.run.smithi003.stdout:033 2023-11-04T05:31:07.994 INFO:teuthology.orchestra.run.smithi003.stdout:033.out 2023-11-04T05:31:07.994 INFO:teuthology.orchestra.run.smithi003.stdout:034 2023-11-04T05:31:07.995 INFO:teuthology.orchestra.run.smithi003.stdout:034.out 2023-11-04T05:31:07.995 INFO:teuthology.orchestra.run.smithi003.stdout:035 2023-11-04T05:31:07.995 INFO:teuthology.orchestra.run.smithi003.stdout:035.cfg 2023-11-04T05:31:07.995 INFO:teuthology.orchestra.run.smithi003.stdout:035.out.default 2023-11-04T05:31:07.995 INFO:teuthology.orchestra.run.smithi003.stdout:035.out.nfs 2023-11-04T05:31:07.996 INFO:teuthology.orchestra.run.smithi003.stdout:036 2023-11-04T05:31:07.996 INFO:teuthology.orchestra.run.smithi003.stdout:036.out 2023-11-04T05:31:07.996 INFO:teuthology.orchestra.run.smithi003.stdout:037 2023-11-04T05:31:07.996 INFO:teuthology.orchestra.run.smithi003.stdout:037.out 2023-11-04T05:31:07.996 INFO:teuthology.orchestra.run.smithi003.stdout:038 2023-11-04T05:31:07.997 INFO:teuthology.orchestra.run.smithi003.stdout:038.out 2023-11-04T05:31:07.997 INFO:teuthology.orchestra.run.smithi003.stdout:039 2023-11-04T05:31:07.997 INFO:teuthology.orchestra.run.smithi003.stdout:039.out 2023-11-04T05:31:07.997 INFO:teuthology.orchestra.run.smithi003.stdout:040 2023-11-04T05:31:07.997 INFO:teuthology.orchestra.run.smithi003.stdout:040.out 2023-11-04T05:31:07.998 INFO:teuthology.orchestra.run.smithi003.stdout:041 2023-11-04T05:31:07.998 INFO:teuthology.orchestra.run.smithi003.stdout:041.out 2023-11-04T05:31:07.998 INFO:teuthology.orchestra.run.smithi003.stdout:042 2023-11-04T05:31:07.998 INFO:teuthology.orchestra.run.smithi003.stdout:042.out 2023-11-04T05:31:07.998 INFO:teuthology.orchestra.run.smithi003.stdout:043 2023-11-04T05:31:07.999 INFO:teuthology.orchestra.run.smithi003.stdout:043.out 2023-11-04T05:31:07.999 INFO:teuthology.orchestra.run.smithi003.stdout:044 2023-11-04T05:31:07.999 INFO:teuthology.orchestra.run.smithi003.stdout:044.out 2023-11-04T05:31:07.999 INFO:teuthology.orchestra.run.smithi003.stdout:045 2023-11-04T05:31:07.999 INFO:teuthology.orchestra.run.smithi003.stdout:045.out 2023-11-04T05:31:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:046 2023-11-04T05:31:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:046.out 2023-11-04T05:31:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:047 2023-11-04T05:31:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:047.out 2023-11-04T05:31:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:048 2023-11-04T05:31:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:048.out 2023-11-04T05:31:08.001 INFO:teuthology.orchestra.run.smithi003.stdout:049 2023-11-04T05:31:08.001 INFO:teuthology.orchestra.run.smithi003.stdout:049.out 2023-11-04T05:31:08.001 INFO:teuthology.orchestra.run.smithi003.stdout:050 2023-11-04T05:31:08.001 INFO:teuthology.orchestra.run.smithi003.stdout:050.cfg 2023-11-04T05:31:08.001 INFO:teuthology.orchestra.run.smithi003.stdout:050.out.default 2023-11-04T05:31:08.002 INFO:teuthology.orchestra.run.smithi003.stdout:050.out.nojournal 2023-11-04T05:31:08.002 INFO:teuthology.orchestra.run.smithi003.stdout:050.out.xfsquota 2023-11-04T05:31:08.002 INFO:teuthology.orchestra.run.smithi003.stdout:051 2023-11-04T05:31:08.002 INFO:teuthology.orchestra.run.smithi003.stdout:051.out 2023-11-04T05:31:08.003 INFO:teuthology.orchestra.run.smithi003.stdout:052 2023-11-04T05:31:08.003 INFO:teuthology.orchestra.run.smithi003.stdout:052.out 2023-11-04T05:31:08.003 INFO:teuthology.orchestra.run.smithi003.stdout:053 2023-11-04T05:31:08.003 INFO:teuthology.orchestra.run.smithi003.stdout:053.out 2023-11-04T05:31:08.003 INFO:teuthology.orchestra.run.smithi003.stdout:054 2023-11-04T05:31:08.003 INFO:teuthology.orchestra.run.smithi003.stdout:054.out 2023-11-04T05:31:08.004 INFO:teuthology.orchestra.run.smithi003.stdout:055 2023-11-04T05:31:08.004 INFO:teuthology.orchestra.run.smithi003.stdout:055.out 2023-11-04T05:31:08.004 INFO:teuthology.orchestra.run.smithi003.stdout:056 2023-11-04T05:31:08.004 INFO:teuthology.orchestra.run.smithi003.stdout:056.out 2023-11-04T05:31:08.004 INFO:teuthology.orchestra.run.smithi003.stdout:057 2023-11-04T05:31:08.005 INFO:teuthology.orchestra.run.smithi003.stdout:057.out 2023-11-04T05:31:08.005 INFO:teuthology.orchestra.run.smithi003.stdout:058 2023-11-04T05:31:08.005 INFO:teuthology.orchestra.run.smithi003.stdout:058.out 2023-11-04T05:31:08.005 INFO:teuthology.orchestra.run.smithi003.stdout:059 2023-11-04T05:31:08.005 INFO:teuthology.orchestra.run.smithi003.stdout:059.out 2023-11-04T05:31:08.006 INFO:teuthology.orchestra.run.smithi003.stdout:060 2023-11-04T05:31:08.006 INFO:teuthology.orchestra.run.smithi003.stdout:060.out 2023-11-04T05:31:08.006 INFO:teuthology.orchestra.run.smithi003.stdout:061 2023-11-04T05:31:08.006 INFO:teuthology.orchestra.run.smithi003.stdout:061.out 2023-11-04T05:31:08.006 INFO:teuthology.orchestra.run.smithi003.stdout:062 2023-11-04T05:31:08.007 INFO:teuthology.orchestra.run.smithi003.stdout:062.out 2023-11-04T05:31:08.007 INFO:teuthology.orchestra.run.smithi003.stdout:063 2023-11-04T05:31:08.007 INFO:teuthology.orchestra.run.smithi003.stdout:063.out 2023-11-04T05:31:08.007 INFO:teuthology.orchestra.run.smithi003.stdout:064 2023-11-04T05:31:08.007 INFO:teuthology.orchestra.run.smithi003.stdout:064.out 2023-11-04T05:31:08.008 INFO:teuthology.orchestra.run.smithi003.stdout:065 2023-11-04T05:31:08.008 INFO:teuthology.orchestra.run.smithi003.stdout:065.out 2023-11-04T05:31:08.008 INFO:teuthology.orchestra.run.smithi003.stdout:066 2023-11-04T05:31:08.008 INFO:teuthology.orchestra.run.smithi003.stdout:066.out 2023-11-04T05:31:08.008 INFO:teuthology.orchestra.run.smithi003.stdout:067 2023-11-04T05:31:08.008 INFO:teuthology.orchestra.run.smithi003.stdout:067.out 2023-11-04T05:31:08.009 INFO:teuthology.orchestra.run.smithi003.stdout:068 2023-11-04T05:31:08.009 INFO:teuthology.orchestra.run.smithi003.stdout:068.out 2023-11-04T05:31:08.009 INFO:teuthology.orchestra.run.smithi003.stdout:069 2023-11-04T05:31:08.009 INFO:teuthology.orchestra.run.smithi003.stdout:069.out 2023-11-04T05:31:08.009 INFO:teuthology.orchestra.run.smithi003.stdout:070 2023-11-04T05:31:08.010 INFO:teuthology.orchestra.run.smithi003.stdout:070.out 2023-11-04T05:31:08.010 INFO:teuthology.orchestra.run.smithi003.stdout:071 2023-11-04T05:31:08.010 INFO:teuthology.orchestra.run.smithi003.stdout:071.out 2023-11-04T05:31:08.010 INFO:teuthology.orchestra.run.smithi003.stdout:072 2023-11-04T05:31:08.010 INFO:teuthology.orchestra.run.smithi003.stdout:072.out 2023-11-04T05:31:08.011 INFO:teuthology.orchestra.run.smithi003.stdout:073 2023-11-04T05:31:08.011 INFO:teuthology.orchestra.run.smithi003.stdout:073.out 2023-11-04T05:31:08.011 INFO:teuthology.orchestra.run.smithi003.stdout:074 2023-11-04T05:31:08.011 INFO:teuthology.orchestra.run.smithi003.stdout:074.out 2023-11-04T05:31:08.011 INFO:teuthology.orchestra.run.smithi003.stdout:075 2023-11-04T05:31:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:075.out 2023-11-04T05:31:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:076 2023-11-04T05:31:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:076.out 2023-11-04T05:31:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:077 2023-11-04T05:31:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:077.out 2023-11-04T05:31:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:078 2023-11-04T05:31:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:078.out 2023-11-04T05:31:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:079 2023-11-04T05:31:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:079.out 2023-11-04T05:31:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:080 2023-11-04T05:31:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:080.out 2023-11-04T05:31:08.014 INFO:teuthology.orchestra.run.smithi003.stdout:081 2023-11-04T05:31:08.014 INFO:teuthology.orchestra.run.smithi003.stdout:081.out 2023-11-04T05:31:08.014 INFO:teuthology.orchestra.run.smithi003.stdout:082 2023-11-04T05:31:08.014 INFO:teuthology.orchestra.run.smithi003.stdout:082.out 2023-11-04T05:31:08.014 INFO:teuthology.orchestra.run.smithi003.stdout:083 2023-11-04T05:31:08.015 INFO:teuthology.orchestra.run.smithi003.stdout:083.out 2023-11-04T05:31:08.015 INFO:teuthology.orchestra.run.smithi003.stdout:084 2023-11-04T05:31:08.015 INFO:teuthology.orchestra.run.smithi003.stdout:084.out 2023-11-04T05:31:08.015 INFO:teuthology.orchestra.run.smithi003.stdout:085 2023-11-04T05:31:08.015 INFO:teuthology.orchestra.run.smithi003.stdout:085.out 2023-11-04T05:31:08.016 INFO:teuthology.orchestra.run.smithi003.stdout:086 2023-11-04T05:31:08.016 INFO:teuthology.orchestra.run.smithi003.stdout:086.out 2023-11-04T05:31:08.016 INFO:teuthology.orchestra.run.smithi003.stdout:087 2023-11-04T05:31:08.016 INFO:teuthology.orchestra.run.smithi003.stdout:087.out 2023-11-04T05:31:08.016 INFO:teuthology.orchestra.run.smithi003.stdout:088 2023-11-04T05:31:08.016 INFO:teuthology.orchestra.run.smithi003.stdout:088.out 2023-11-04T05:31:08.017 INFO:teuthology.orchestra.run.smithi003.stdout:089 2023-11-04T05:31:08.017 INFO:teuthology.orchestra.run.smithi003.stdout:089.out 2023-11-04T05:31:08.017 INFO:teuthology.orchestra.run.smithi003.stdout:090 2023-11-04T05:31:08.017 INFO:teuthology.orchestra.run.smithi003.stdout:090.out 2023-11-04T05:31:08.017 INFO:teuthology.orchestra.run.smithi003.stdout:091 2023-11-04T05:31:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:091.out 2023-11-04T05:31:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:092 2023-11-04T05:31:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:092.out 2023-11-04T05:31:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:093 2023-11-04T05:31:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:093.out 2023-11-04T05:31:08.019 INFO:teuthology.orchestra.run.smithi003.stdout:094 2023-11-04T05:31:08.019 INFO:teuthology.orchestra.run.smithi003.stdout:094.out 2023-11-04T05:31:08.019 INFO:teuthology.orchestra.run.smithi003.stdout:095 2023-11-04T05:31:08.019 INFO:teuthology.orchestra.run.smithi003.stdout:095.out 2023-11-04T05:31:08.019 INFO:teuthology.orchestra.run.smithi003.stdout:096 2023-11-04T05:31:08.019 INFO:teuthology.orchestra.run.smithi003.stdout:096.out 2023-11-04T05:31:08.020 INFO:teuthology.orchestra.run.smithi003.stdout:097 2023-11-04T05:31:08.020 INFO:teuthology.orchestra.run.smithi003.stdout:097.out 2023-11-04T05:31:08.020 INFO:teuthology.orchestra.run.smithi003.stdout:098 2023-11-04T05:31:08.020 INFO:teuthology.orchestra.run.smithi003.stdout:098.out 2023-11-04T05:31:08.020 INFO:teuthology.orchestra.run.smithi003.stdout:099 2023-11-04T05:31:08.021 INFO:teuthology.orchestra.run.smithi003.stdout:099.out 2023-11-04T05:31:08.021 INFO:teuthology.orchestra.run.smithi003.stdout:100 2023-11-04T05:31:08.021 INFO:teuthology.orchestra.run.smithi003.stdout:100.out 2023-11-04T05:31:08.021 INFO:teuthology.orchestra.run.smithi003.stdout:101 2023-11-04T05:31:08.021 INFO:teuthology.orchestra.run.smithi003.stdout:101.out 2023-11-04T05:31:08.022 INFO:teuthology.orchestra.run.smithi003.stdout:102 2023-11-04T05:31:08.022 INFO:teuthology.orchestra.run.smithi003.stdout:102.out 2023-11-04T05:31:08.022 INFO:teuthology.orchestra.run.smithi003.stdout:103 2023-11-04T05:31:08.022 INFO:teuthology.orchestra.run.smithi003.stdout:103.out 2023-11-04T05:31:08.022 INFO:teuthology.orchestra.run.smithi003.stdout:104 2023-11-04T05:31:08.022 INFO:teuthology.orchestra.run.smithi003.stdout:104.out 2023-11-04T05:31:08.023 INFO:teuthology.orchestra.run.smithi003.stdout:105 2023-11-04T05:31:08.023 INFO:teuthology.orchestra.run.smithi003.stdout:105.out 2023-11-04T05:31:08.023 INFO:teuthology.orchestra.run.smithi003.stdout:106 2023-11-04T05:31:08.023 INFO:teuthology.orchestra.run.smithi003.stdout:106.out 2023-11-04T05:31:08.023 INFO:teuthology.orchestra.run.smithi003.stdout:107 2023-11-04T05:31:08.024 INFO:teuthology.orchestra.run.smithi003.stdout:107.out 2023-11-04T05:31:08.024 INFO:teuthology.orchestra.run.smithi003.stdout:108 2023-11-04T05:31:08.024 INFO:teuthology.orchestra.run.smithi003.stdout:108.out 2023-11-04T05:31:08.024 INFO:teuthology.orchestra.run.smithi003.stdout:109 2023-11-04T05:31:08.024 INFO:teuthology.orchestra.run.smithi003.stdout:109.out 2023-11-04T05:31:08.025 INFO:teuthology.orchestra.run.smithi003.stdout:110 2023-11-04T05:31:08.025 INFO:teuthology.orchestra.run.smithi003.stdout:110.out 2023-11-04T05:31:08.025 INFO:teuthology.orchestra.run.smithi003.stdout:111 2023-11-04T05:31:08.025 INFO:teuthology.orchestra.run.smithi003.stdout:111.out 2023-11-04T05:31:08.025 INFO:teuthology.orchestra.run.smithi003.stdout:112 2023-11-04T05:31:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:112.out 2023-11-04T05:31:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:113 2023-11-04T05:31:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:113.out 2023-11-04T05:31:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:114 2023-11-04T05:31:08.026 INFO:teuthology.orchestra.run.smithi003.stdout:114.out 2023-11-04T05:31:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:115 2023-11-04T05:31:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:115.out 2023-11-04T05:31:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:116 2023-11-04T05:31:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:116.out 2023-11-04T05:31:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:117 2023-11-04T05:31:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:117.out 2023-11-04T05:31:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:118 2023-11-04T05:31:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:118.out 2023-11-04T05:31:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:119 2023-11-04T05:31:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:119.out 2023-11-04T05:31:08.029 INFO:teuthology.orchestra.run.smithi003.stdout:120 2023-11-04T05:31:08.029 INFO:teuthology.orchestra.run.smithi003.stdout:120.out 2023-11-04T05:31:08.029 INFO:teuthology.orchestra.run.smithi003.stdout:121 2023-11-04T05:31:08.029 INFO:teuthology.orchestra.run.smithi003.stdout:121.out 2023-11-04T05:31:08.029 INFO:teuthology.orchestra.run.smithi003.stdout:122 2023-11-04T05:31:08.030 INFO:teuthology.orchestra.run.smithi003.stdout:122.out 2023-11-04T05:31:08.030 INFO:teuthology.orchestra.run.smithi003.stdout:123 2023-11-04T05:31:08.030 INFO:teuthology.orchestra.run.smithi003.stdout:123.out 2023-11-04T05:31:08.030 INFO:teuthology.orchestra.run.smithi003.stdout:124 2023-11-04T05:31:08.030 INFO:teuthology.orchestra.run.smithi003.stdout:124.out 2023-11-04T05:31:08.031 INFO:teuthology.orchestra.run.smithi003.stdout:125 2023-11-04T05:31:08.031 INFO:teuthology.orchestra.run.smithi003.stdout:125.out 2023-11-04T05:31:08.031 INFO:teuthology.orchestra.run.smithi003.stdout:126 2023-11-04T05:31:08.031 INFO:teuthology.orchestra.run.smithi003.stdout:126.out 2023-11-04T05:31:08.031 INFO:teuthology.orchestra.run.smithi003.stdout:127 2023-11-04T05:31:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:127.out 2023-11-04T05:31:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:128 2023-11-04T05:31:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:128.out 2023-11-04T05:31:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:129 2023-11-04T05:31:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:129.out 2023-11-04T05:31:08.032 INFO:teuthology.orchestra.run.smithi003.stdout:130 2023-11-04T05:31:08.033 INFO:teuthology.orchestra.run.smithi003.stdout:130.out 2023-11-04T05:31:08.033 INFO:teuthology.orchestra.run.smithi003.stdout:131 2023-11-04T05:31:08.033 INFO:teuthology.orchestra.run.smithi003.stdout:131.out 2023-11-04T05:31:08.033 INFO:teuthology.orchestra.run.smithi003.stdout:132 2023-11-04T05:31:08.033 INFO:teuthology.orchestra.run.smithi003.stdout:132.out 2023-11-04T05:31:08.034 INFO:teuthology.orchestra.run.smithi003.stdout:133 2023-11-04T05:31:08.034 INFO:teuthology.orchestra.run.smithi003.stdout:133.out 2023-11-04T05:31:08.034 INFO:teuthology.orchestra.run.smithi003.stdout:134 2023-11-04T05:31:08.034 INFO:teuthology.orchestra.run.smithi003.stdout:134.out 2023-11-04T05:31:08.034 INFO:teuthology.orchestra.run.smithi003.stdout:135 2023-11-04T05:31:08.035 INFO:teuthology.orchestra.run.smithi003.stdout:135.out 2023-11-04T05:31:08.035 INFO:teuthology.orchestra.run.smithi003.stdout:136 2023-11-04T05:31:08.035 INFO:teuthology.orchestra.run.smithi003.stdout:136.out 2023-11-04T05:31:08.035 INFO:teuthology.orchestra.run.smithi003.stdout:137 2023-11-04T05:31:08.035 INFO:teuthology.orchestra.run.smithi003.stdout:137.out 2023-11-04T05:31:08.036 INFO:teuthology.orchestra.run.smithi003.stdout:138 2023-11-04T05:31:08.036 INFO:teuthology.orchestra.run.smithi003.stdout:138.out 2023-11-04T05:31:08.036 INFO:teuthology.orchestra.run.smithi003.stdout:139 2023-11-04T05:31:08.036 INFO:teuthology.orchestra.run.smithi003.stdout:139.out 2023-11-04T05:31:08.036 INFO:teuthology.orchestra.run.smithi003.stdout:140 2023-11-04T05:31:08.037 INFO:teuthology.orchestra.run.smithi003.stdout:140.out 2023-11-04T05:31:08.037 INFO:teuthology.orchestra.run.smithi003.stdout:141 2023-11-04T05:31:08.037 INFO:teuthology.orchestra.run.smithi003.stdout:141.out 2023-11-04T05:31:08.037 INFO:teuthology.orchestra.run.smithi003.stdout:142 2023-11-04T05:31:08.037 INFO:teuthology.orchestra.run.smithi003.stdout:142.out 2023-11-04T05:31:08.037 INFO:teuthology.orchestra.run.smithi003.stdout:143 2023-11-04T05:31:08.038 INFO:teuthology.orchestra.run.smithi003.stdout:143.out 2023-11-04T05:31:08.038 INFO:teuthology.orchestra.run.smithi003.stdout:144 2023-11-04T05:31:08.038 INFO:teuthology.orchestra.run.smithi003.stdout:144.out 2023-11-04T05:31:08.038 INFO:teuthology.orchestra.run.smithi003.stdout:145 2023-11-04T05:31:08.038 INFO:teuthology.orchestra.run.smithi003.stdout:145.out 2023-11-04T05:31:08.039 INFO:teuthology.orchestra.run.smithi003.stdout:146 2023-11-04T05:31:08.039 INFO:teuthology.orchestra.run.smithi003.stdout:146.out 2023-11-04T05:31:08.039 INFO:teuthology.orchestra.run.smithi003.stdout:147 2023-11-04T05:31:08.039 INFO:teuthology.orchestra.run.smithi003.stdout:147.out 2023-11-04T05:31:08.039 INFO:teuthology.orchestra.run.smithi003.stdout:148 2023-11-04T05:31:08.040 INFO:teuthology.orchestra.run.smithi003.stdout:148.out 2023-11-04T05:31:08.040 INFO:teuthology.orchestra.run.smithi003.stdout:149 2023-11-04T05:31:08.040 INFO:teuthology.orchestra.run.smithi003.stdout:149.out 2023-11-04T05:31:08.040 INFO:teuthology.orchestra.run.smithi003.stdout:150 2023-11-04T05:31:08.040 INFO:teuthology.orchestra.run.smithi003.stdout:150.out 2023-11-04T05:31:08.041 INFO:teuthology.orchestra.run.smithi003.stdout:151 2023-11-04T05:31:08.041 INFO:teuthology.orchestra.run.smithi003.stdout:151.out 2023-11-04T05:31:08.041 INFO:teuthology.orchestra.run.smithi003.stdout:152 2023-11-04T05:31:08.041 INFO:teuthology.orchestra.run.smithi003.stdout:152.out 2023-11-04T05:31:08.041 INFO:teuthology.orchestra.run.smithi003.stdout:153 2023-11-04T05:31:08.042 INFO:teuthology.orchestra.run.smithi003.stdout:153.out 2023-11-04T05:31:08.042 INFO:teuthology.orchestra.run.smithi003.stdout:154 2023-11-04T05:31:08.042 INFO:teuthology.orchestra.run.smithi003.stdout:154.out 2023-11-04T05:31:08.042 INFO:teuthology.orchestra.run.smithi003.stdout:155 2023-11-04T05:31:08.042 INFO:teuthology.orchestra.run.smithi003.stdout:155.out 2023-11-04T05:31:08.043 INFO:teuthology.orchestra.run.smithi003.stdout:156 2023-11-04T05:31:08.043 INFO:teuthology.orchestra.run.smithi003.stdout:156.out 2023-11-04T05:31:08.043 INFO:teuthology.orchestra.run.smithi003.stdout:157 2023-11-04T05:31:08.043 INFO:teuthology.orchestra.run.smithi003.stdout:157.out 2023-11-04T05:31:08.043 INFO:teuthology.orchestra.run.smithi003.stdout:158 2023-11-04T05:31:08.044 INFO:teuthology.orchestra.run.smithi003.stdout:158.out 2023-11-04T05:31:08.044 INFO:teuthology.orchestra.run.smithi003.stdout:159 2023-11-04T05:31:08.044 INFO:teuthology.orchestra.run.smithi003.stdout:159.out 2023-11-04T05:31:08.044 INFO:teuthology.orchestra.run.smithi003.stdout:160 2023-11-04T05:31:08.044 INFO:teuthology.orchestra.run.smithi003.stdout:160.out 2023-11-04T05:31:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:161 2023-11-04T05:31:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:161.out 2023-11-04T05:31:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:162 2023-11-04T05:31:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:162.out 2023-11-04T05:31:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:163 2023-11-04T05:31:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:163.out 2023-11-04T05:31:08.046 INFO:teuthology.orchestra.run.smithi003.stdout:164 2023-11-04T05:31:08.046 INFO:teuthology.orchestra.run.smithi003.stdout:164.out 2023-11-04T05:31:08.046 INFO:teuthology.orchestra.run.smithi003.stdout:165 2023-11-04T05:31:08.046 INFO:teuthology.orchestra.run.smithi003.stdout:165.out 2023-11-04T05:31:08.046 INFO:teuthology.orchestra.run.smithi003.stdout:166 2023-11-04T05:31:08.047 INFO:teuthology.orchestra.run.smithi003.stdout:166.out 2023-11-04T05:31:08.047 INFO:teuthology.orchestra.run.smithi003.stdout:167 2023-11-04T05:31:08.047 INFO:teuthology.orchestra.run.smithi003.stdout:167.out 2023-11-04T05:31:08.047 INFO:teuthology.orchestra.run.smithi003.stdout:168 2023-11-04T05:31:08.047 INFO:teuthology.orchestra.run.smithi003.stdout:168.out 2023-11-04T05:31:08.048 INFO:teuthology.orchestra.run.smithi003.stdout:169 2023-11-04T05:31:08.048 INFO:teuthology.orchestra.run.smithi003.stdout:169.out 2023-11-04T05:31:08.048 INFO:teuthology.orchestra.run.smithi003.stdout:170 2023-11-04T05:31:08.048 INFO:teuthology.orchestra.run.smithi003.stdout:170.out 2023-11-04T05:31:08.048 INFO:teuthology.orchestra.run.smithi003.stdout:171 2023-11-04T05:31:08.049 INFO:teuthology.orchestra.run.smithi003.stdout:171.out 2023-11-04T05:31:08.049 INFO:teuthology.orchestra.run.smithi003.stdout:172 2023-11-04T05:31:08.049 INFO:teuthology.orchestra.run.smithi003.stdout:172.out 2023-11-04T05:31:08.049 INFO:teuthology.orchestra.run.smithi003.stdout:173 2023-11-04T05:31:08.049 INFO:teuthology.orchestra.run.smithi003.stdout:173.out 2023-11-04T05:31:08.050 INFO:teuthology.orchestra.run.smithi003.stdout:174 2023-11-04T05:31:08.050 INFO:teuthology.orchestra.run.smithi003.stdout:174.out 2023-11-04T05:31:08.050 INFO:teuthology.orchestra.run.smithi003.stdout:175 2023-11-04T05:31:08.050 INFO:teuthology.orchestra.run.smithi003.stdout:175.out 2023-11-04T05:31:08.050 INFO:teuthology.orchestra.run.smithi003.stdout:176 2023-11-04T05:31:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:176.out 2023-11-04T05:31:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:177 2023-11-04T05:31:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:177.out 2023-11-04T05:31:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:178 2023-11-04T05:31:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:178.out 2023-11-04T05:31:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:179 2023-11-04T05:31:08.052 INFO:teuthology.orchestra.run.smithi003.stdout:179.out 2023-11-04T05:31:08.052 INFO:teuthology.orchestra.run.smithi003.stdout:180 2023-11-04T05:31:08.052 INFO:teuthology.orchestra.run.smithi003.stdout:180.out 2023-11-04T05:31:08.052 INFO:teuthology.orchestra.run.smithi003.stdout:181 2023-11-04T05:31:08.052 INFO:teuthology.orchestra.run.smithi003.stdout:181.out 2023-11-04T05:31:08.053 INFO:teuthology.orchestra.run.smithi003.stdout:182 2023-11-04T05:31:08.053 INFO:teuthology.orchestra.run.smithi003.stdout:182.out 2023-11-04T05:31:08.053 INFO:teuthology.orchestra.run.smithi003.stdout:183 2023-11-04T05:31:08.053 INFO:teuthology.orchestra.run.smithi003.stdout:183.out 2023-11-04T05:31:08.053 INFO:teuthology.orchestra.run.smithi003.stdout:184 2023-11-04T05:31:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:184.out 2023-11-04T05:31:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:185 2023-11-04T05:31:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:185.out 2023-11-04T05:31:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:186 2023-11-04T05:31:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:186.out 2023-11-04T05:31:08.055 INFO:teuthology.orchestra.run.smithi003.stdout:187 2023-11-04T05:31:08.055 INFO:teuthology.orchestra.run.smithi003.stdout:187.out 2023-11-04T05:31:08.055 INFO:teuthology.orchestra.run.smithi003.stdout:188 2023-11-04T05:31:08.055 INFO:teuthology.orchestra.run.smithi003.stdout:188.out 2023-11-04T05:31:08.055 INFO:teuthology.orchestra.run.smithi003.stdout:189 2023-11-04T05:31:08.055 INFO:teuthology.orchestra.run.smithi003.stdout:189.out 2023-11-04T05:31:08.056 INFO:teuthology.orchestra.run.smithi003.stdout:190 2023-11-04T05:31:08.056 INFO:teuthology.orchestra.run.smithi003.stdout:190.out 2023-11-04T05:31:08.056 INFO:teuthology.orchestra.run.smithi003.stdout:191 2023-11-04T05:31:08.056 INFO:teuthology.orchestra.run.smithi003.stdout:191.out 2023-11-04T05:31:08.056 INFO:teuthology.orchestra.run.smithi003.stdout:192 2023-11-04T05:31:08.057 INFO:teuthology.orchestra.run.smithi003.stdout:192.out 2023-11-04T05:31:08.057 INFO:teuthology.orchestra.run.smithi003.stdout:193 2023-11-04T05:31:08.057 INFO:teuthology.orchestra.run.smithi003.stdout:193.out 2023-11-04T05:31:08.057 INFO:teuthology.orchestra.run.smithi003.stdout:194 2023-11-04T05:31:08.057 INFO:teuthology.orchestra.run.smithi003.stdout:194.out 2023-11-04T05:31:08.058 INFO:teuthology.orchestra.run.smithi003.stdout:195 2023-11-04T05:31:08.058 INFO:teuthology.orchestra.run.smithi003.stdout:195.out 2023-11-04T05:31:08.058 INFO:teuthology.orchestra.run.smithi003.stdout:196 2023-11-04T05:31:08.058 INFO:teuthology.orchestra.run.smithi003.stdout:196.out 2023-11-04T05:31:08.058 INFO:teuthology.orchestra.run.smithi003.stdout:197 2023-11-04T05:31:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:197.out 2023-11-04T05:31:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:198 2023-11-04T05:31:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:198.out 2023-11-04T05:31:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:199 2023-11-04T05:31:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:199.out 2023-11-04T05:31:08.060 INFO:teuthology.orchestra.run.smithi003.stdout:200 2023-11-04T05:31:08.060 INFO:teuthology.orchestra.run.smithi003.stdout:200.out 2023-11-04T05:31:08.060 INFO:teuthology.orchestra.run.smithi003.stdout:201 2023-11-04T05:31:08.060 INFO:teuthology.orchestra.run.smithi003.stdout:201.out 2023-11-04T05:31:08.060 INFO:teuthology.orchestra.run.smithi003.stdout:202 2023-11-04T05:31:08.061 INFO:teuthology.orchestra.run.smithi003.stdout:202.out 2023-11-04T05:31:08.061 INFO:teuthology.orchestra.run.smithi003.stdout:203 2023-11-04T05:31:08.061 INFO:teuthology.orchestra.run.smithi003.stdout:203.out 2023-11-04T05:31:08.061 INFO:teuthology.orchestra.run.smithi003.stdout:204 2023-11-04T05:31:08.061 INFO:teuthology.orchestra.run.smithi003.stdout:204.out 2023-11-04T05:31:08.061 INFO:teuthology.orchestra.run.smithi003.stdout:205 2023-11-04T05:31:08.062 INFO:teuthology.orchestra.run.smithi003.stdout:205.out 2023-11-04T05:31:08.062 INFO:teuthology.orchestra.run.smithi003.stdout:206 2023-11-04T05:31:08.062 INFO:teuthology.orchestra.run.smithi003.stdout:206.out 2023-11-04T05:31:08.062 INFO:teuthology.orchestra.run.smithi003.stdout:207 2023-11-04T05:31:08.062 INFO:teuthology.orchestra.run.smithi003.stdout:207.out 2023-11-04T05:31:08.063 INFO:teuthology.orchestra.run.smithi003.stdout:208 2023-11-04T05:31:08.063 INFO:teuthology.orchestra.run.smithi003.stdout:208.out 2023-11-04T05:31:08.063 INFO:teuthology.orchestra.run.smithi003.stdout:209 2023-11-04T05:31:08.063 INFO:teuthology.orchestra.run.smithi003.stdout:209.out 2023-11-04T05:31:08.063 INFO:teuthology.orchestra.run.smithi003.stdout:210 2023-11-04T05:31:08.064 INFO:teuthology.orchestra.run.smithi003.stdout:210.out 2023-11-04T05:31:08.064 INFO:teuthology.orchestra.run.smithi003.stdout:211 2023-11-04T05:31:08.064 INFO:teuthology.orchestra.run.smithi003.stdout:211.out 2023-11-04T05:31:08.064 INFO:teuthology.orchestra.run.smithi003.stdout:212 2023-11-04T05:31:08.064 INFO:teuthology.orchestra.run.smithi003.stdout:212.out 2023-11-04T05:31:08.065 INFO:teuthology.orchestra.run.smithi003.stdout:213 2023-11-04T05:31:08.065 INFO:teuthology.orchestra.run.smithi003.stdout:213.out 2023-11-04T05:31:08.065 INFO:teuthology.orchestra.run.smithi003.stdout:214 2023-11-04T05:31:08.065 INFO:teuthology.orchestra.run.smithi003.stdout:214.out 2023-11-04T05:31:08.065 INFO:teuthology.orchestra.run.smithi003.stdout:215 2023-11-04T05:31:08.065 INFO:teuthology.orchestra.run.smithi003.stdout:215.out 2023-11-04T05:31:08.066 INFO:teuthology.orchestra.run.smithi003.stdout:216 2023-11-04T05:31:08.066 INFO:teuthology.orchestra.run.smithi003.stdout:216.out 2023-11-04T05:31:08.066 INFO:teuthology.orchestra.run.smithi003.stdout:217 2023-11-04T05:31:08.066 INFO:teuthology.orchestra.run.smithi003.stdout:217.out 2023-11-04T05:31:08.066 INFO:teuthology.orchestra.run.smithi003.stdout:218 2023-11-04T05:31:08.067 INFO:teuthology.orchestra.run.smithi003.stdout:218.out 2023-11-04T05:31:08.067 INFO:teuthology.orchestra.run.smithi003.stdout:219 2023-11-04T05:31:08.067 INFO:teuthology.orchestra.run.smithi003.stdout:219.out 2023-11-04T05:31:08.067 INFO:teuthology.orchestra.run.smithi003.stdout:220 2023-11-04T05:31:08.067 INFO:teuthology.orchestra.run.smithi003.stdout:220.out 2023-11-04T05:31:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:221 2023-11-04T05:31:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:221.out 2023-11-04T05:31:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:222 2023-11-04T05:31:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:222.out 2023-11-04T05:31:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:223 2023-11-04T05:31:08.068 INFO:teuthology.orchestra.run.smithi003.stdout:223.out 2023-11-04T05:31:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:224 2023-11-04T05:31:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:224.out 2023-11-04T05:31:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:225 2023-11-04T05:31:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:225.out 2023-11-04T05:31:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:226 2023-11-04T05:31:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:226.out 2023-11-04T05:31:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:227 2023-11-04T05:31:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:227.out 2023-11-04T05:31:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:228 2023-11-04T05:31:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:228.out 2023-11-04T05:31:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:229 2023-11-04T05:31:08.071 INFO:teuthology.orchestra.run.smithi003.stdout:229.out 2023-11-04T05:31:08.071 INFO:teuthology.orchestra.run.smithi003.stdout:230 2023-11-04T05:31:08.071 INFO:teuthology.orchestra.run.smithi003.stdout:230.out 2023-11-04T05:31:08.071 INFO:teuthology.orchestra.run.smithi003.stdout:231 2023-11-04T05:31:08.071 INFO:teuthology.orchestra.run.smithi003.stdout:231.out 2023-11-04T05:31:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:232 2023-11-04T05:31:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:232.out 2023-11-04T05:31:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:233 2023-11-04T05:31:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:233.out 2023-11-04T05:31:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:234 2023-11-04T05:31:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:234.out 2023-11-04T05:31:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:235 2023-11-04T05:31:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:235.out 2023-11-04T05:31:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:236 2023-11-04T05:31:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:236.out 2023-11-04T05:31:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:237 2023-11-04T05:31:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:237.out 2023-11-04T05:31:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:238 2023-11-04T05:31:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:238.out 2023-11-04T05:31:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:239 2023-11-04T05:31:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:239.out 2023-11-04T05:31:08.075 INFO:teuthology.orchestra.run.smithi003.stdout:240 2023-11-04T05:31:08.075 INFO:teuthology.orchestra.run.smithi003.stdout:240.out 2023-11-04T05:31:08.075 INFO:teuthology.orchestra.run.smithi003.stdout:241 2023-11-04T05:31:08.075 INFO:teuthology.orchestra.run.smithi003.stdout:241.out 2023-11-04T05:31:08.075 INFO:teuthology.orchestra.run.smithi003.stdout:242 2023-11-04T05:31:08.077 INFO:teuthology.orchestra.run.smithi003.stdout:242.out 2023-11-04T05:31:08.077 INFO:teuthology.orchestra.run.smithi003.stdout:243 2023-11-04T05:31:08.077 INFO:teuthology.orchestra.run.smithi003.stdout:243.out 2023-11-04T05:31:08.077 INFO:teuthology.orchestra.run.smithi003.stdout:244 2023-11-04T05:31:08.078 INFO:teuthology.orchestra.run.smithi003.stdout:244.out 2023-11-04T05:31:08.078 INFO:teuthology.orchestra.run.smithi003.stdout:245 2023-11-04T05:31:08.078 INFO:teuthology.orchestra.run.smithi003.stdout:245.out 2023-11-04T05:31:08.078 INFO:teuthology.orchestra.run.smithi003.stdout:246 2023-11-04T05:31:08.078 INFO:teuthology.orchestra.run.smithi003.stdout:246.out 2023-11-04T05:31:08.079 INFO:teuthology.orchestra.run.smithi003.stdout:247 2023-11-04T05:31:08.079 INFO:teuthology.orchestra.run.smithi003.stdout:247.out 2023-11-04T05:31:08.079 INFO:teuthology.orchestra.run.smithi003.stdout:248 2023-11-04T05:31:08.079 INFO:teuthology.orchestra.run.smithi003.stdout:248.out 2023-11-04T05:31:08.079 INFO:teuthology.orchestra.run.smithi003.stdout:249 2023-11-04T05:31:08.080 INFO:teuthology.orchestra.run.smithi003.stdout:249.out 2023-11-04T05:31:08.080 INFO:teuthology.orchestra.run.smithi003.stdout:250 2023-11-04T05:31:08.080 INFO:teuthology.orchestra.run.smithi003.stdout:250.out 2023-11-04T05:31:08.080 INFO:teuthology.orchestra.run.smithi003.stdout:251 2023-11-04T05:31:08.080 INFO:teuthology.orchestra.run.smithi003.stdout:251.out 2023-11-04T05:31:08.080 INFO:teuthology.orchestra.run.smithi003.stdout:252 2023-11-04T05:31:08.081 INFO:teuthology.orchestra.run.smithi003.stdout:252.out 2023-11-04T05:31:08.081 INFO:teuthology.orchestra.run.smithi003.stdout:253 2023-11-04T05:31:08.081 INFO:teuthology.orchestra.run.smithi003.stdout:253.out 2023-11-04T05:31:08.081 INFO:teuthology.orchestra.run.smithi003.stdout:254 2023-11-04T05:31:08.081 INFO:teuthology.orchestra.run.smithi003.stdout:254.out 2023-11-04T05:31:08.082 INFO:teuthology.orchestra.run.smithi003.stdout:255 2023-11-04T05:31:08.082 INFO:teuthology.orchestra.run.smithi003.stdout:255.out 2023-11-04T05:31:08.082 INFO:teuthology.orchestra.run.smithi003.stdout:256 2023-11-04T05:31:08.082 INFO:teuthology.orchestra.run.smithi003.stdout:256.out 2023-11-04T05:31:08.082 INFO:teuthology.orchestra.run.smithi003.stdout:257 2023-11-04T05:31:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:257.out 2023-11-04T05:31:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:258 2023-11-04T05:31:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:258.out 2023-11-04T05:31:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:259 2023-11-04T05:31:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:259.out 2023-11-04T05:31:08.083 INFO:teuthology.orchestra.run.smithi003.stdout:260 2023-11-04T05:31:08.084 INFO:teuthology.orchestra.run.smithi003.stdout:260.out 2023-11-04T05:31:08.084 INFO:teuthology.orchestra.run.smithi003.stdout:261 2023-11-04T05:31:08.084 INFO:teuthology.orchestra.run.smithi003.stdout:261.out 2023-11-04T05:31:08.084 INFO:teuthology.orchestra.run.smithi003.stdout:262 2023-11-04T05:31:08.084 INFO:teuthology.orchestra.run.smithi003.stdout:262.out 2023-11-04T05:31:08.085 INFO:teuthology.orchestra.run.smithi003.stdout:263 2023-11-04T05:31:08.085 INFO:teuthology.orchestra.run.smithi003.stdout:263.out 2023-11-04T05:31:08.085 INFO:teuthology.orchestra.run.smithi003.stdout:264 2023-11-04T05:31:08.085 INFO:teuthology.orchestra.run.smithi003.stdout:264.out 2023-11-04T05:31:08.085 INFO:teuthology.orchestra.run.smithi003.stdout:265 2023-11-04T05:31:08.086 INFO:teuthology.orchestra.run.smithi003.stdout:265.out 2023-11-04T05:31:08.086 INFO:teuthology.orchestra.run.smithi003.stdout:266 2023-11-04T05:31:08.086 INFO:teuthology.orchestra.run.smithi003.stdout:266.out 2023-11-04T05:31:08.086 INFO:teuthology.orchestra.run.smithi003.stdout:267 2023-11-04T05:31:08.086 INFO:teuthology.orchestra.run.smithi003.stdout:267.out 2023-11-04T05:31:08.087 INFO:teuthology.orchestra.run.smithi003.stdout:268 2023-11-04T05:31:08.087 INFO:teuthology.orchestra.run.smithi003.stdout:268.out 2023-11-04T05:31:08.087 INFO:teuthology.orchestra.run.smithi003.stdout:269 2023-11-04T05:31:08.087 INFO:teuthology.orchestra.run.smithi003.stdout:269.out 2023-11-04T05:31:08.087 INFO:teuthology.orchestra.run.smithi003.stdout:270 2023-11-04T05:31:08.088 INFO:teuthology.orchestra.run.smithi003.stdout:270.out 2023-11-04T05:31:08.088 INFO:teuthology.orchestra.run.smithi003.stdout:271 2023-11-04T05:31:08.088 INFO:teuthology.orchestra.run.smithi003.stdout:271.out 2023-11-04T05:31:08.088 INFO:teuthology.orchestra.run.smithi003.stdout:272 2023-11-04T05:31:08.088 INFO:teuthology.orchestra.run.smithi003.stdout:272.out 2023-11-04T05:31:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:273 2023-11-04T05:31:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:273.out 2023-11-04T05:31:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:274 2023-11-04T05:31:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:274.out 2023-11-04T05:31:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:275 2023-11-04T05:31:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:275.out 2023-11-04T05:31:08.090 INFO:teuthology.orchestra.run.smithi003.stdout:276 2023-11-04T05:31:08.090 INFO:teuthology.orchestra.run.smithi003.stdout:276.out 2023-11-04T05:31:08.090 INFO:teuthology.orchestra.run.smithi003.stdout:277 2023-11-04T05:31:08.090 INFO:teuthology.orchestra.run.smithi003.stdout:277.out 2023-11-04T05:31:08.090 INFO:teuthology.orchestra.run.smithi003.stdout:278 2023-11-04T05:31:08.091 INFO:teuthology.orchestra.run.smithi003.stdout:278.out 2023-11-04T05:31:08.091 INFO:teuthology.orchestra.run.smithi003.stdout:279 2023-11-04T05:31:08.091 INFO:teuthology.orchestra.run.smithi003.stdout:279.out 2023-11-04T05:31:08.091 INFO:teuthology.orchestra.run.smithi003.stdout:280 2023-11-04T05:31:08.091 INFO:teuthology.orchestra.run.smithi003.stdout:280.out 2023-11-04T05:31:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:281 2023-11-04T05:31:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:281.out 2023-11-04T05:31:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:282 2023-11-04T05:31:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:282.out 2023-11-04T05:31:08.092 INFO:teuthology.orchestra.run.smithi003.stdout:283 2023-11-04T05:31:08.093 INFO:teuthology.orchestra.run.smithi003.stdout:283.out 2023-11-04T05:31:08.093 INFO:teuthology.orchestra.run.smithi003.stdout:284 2023-11-04T05:31:08.093 INFO:teuthology.orchestra.run.smithi003.stdout:284.out 2023-11-04T05:31:08.093 INFO:teuthology.orchestra.run.smithi003.stdout:285 2023-11-04T05:31:08.093 INFO:teuthology.orchestra.run.smithi003.stdout:285.out 2023-11-04T05:31:08.093 INFO:teuthology.orchestra.run.smithi003.stdout:286 2023-11-04T05:31:08.094 INFO:teuthology.orchestra.run.smithi003.stdout:286.out 2023-11-04T05:31:08.094 INFO:teuthology.orchestra.run.smithi003.stdout:287 2023-11-04T05:31:08.094 INFO:teuthology.orchestra.run.smithi003.stdout:287.out 2023-11-04T05:31:08.094 INFO:teuthology.orchestra.run.smithi003.stdout:288 2023-11-04T05:31:08.094 INFO:teuthology.orchestra.run.smithi003.stdout:288.out 2023-11-04T05:31:08.095 INFO:teuthology.orchestra.run.smithi003.stdout:289 2023-11-04T05:31:08.095 INFO:teuthology.orchestra.run.smithi003.stdout:289.out 2023-11-04T05:31:08.095 INFO:teuthology.orchestra.run.smithi003.stdout:290 2023-11-04T05:31:08.095 INFO:teuthology.orchestra.run.smithi003.stdout:290.out 2023-11-04T05:31:08.095 INFO:teuthology.orchestra.run.smithi003.stdout:291 2023-11-04T05:31:08.096 INFO:teuthology.orchestra.run.smithi003.stdout:291.out 2023-11-04T05:31:08.096 INFO:teuthology.orchestra.run.smithi003.stdout:292 2023-11-04T05:31:08.096 INFO:teuthology.orchestra.run.smithi003.stdout:292.out 2023-11-04T05:31:08.096 INFO:teuthology.orchestra.run.smithi003.stdout:293 2023-11-04T05:31:08.096 INFO:teuthology.orchestra.run.smithi003.stdout:293.out 2023-11-04T05:31:08.096 INFO:teuthology.orchestra.run.smithi003.stdout:294 2023-11-04T05:31:08.097 INFO:teuthology.orchestra.run.smithi003.stdout:294.out 2023-11-04T05:31:08.097 INFO:teuthology.orchestra.run.smithi003.stdout:295 2023-11-04T05:31:08.097 INFO:teuthology.orchestra.run.smithi003.stdout:295.out 2023-11-04T05:31:08.097 INFO:teuthology.orchestra.run.smithi003.stdout:296 2023-11-04T05:31:08.097 INFO:teuthology.orchestra.run.smithi003.stdout:296.out 2023-11-04T05:31:08.098 INFO:teuthology.orchestra.run.smithi003.stdout:297 2023-11-04T05:31:08.098 INFO:teuthology.orchestra.run.smithi003.stdout:297.out 2023-11-04T05:31:08.098 INFO:teuthology.orchestra.run.smithi003.stdout:298 2023-11-04T05:31:08.098 INFO:teuthology.orchestra.run.smithi003.stdout:298.out 2023-11-04T05:31:08.098 INFO:teuthology.orchestra.run.smithi003.stdout:299 2023-11-04T05:31:08.099 INFO:teuthology.orchestra.run.smithi003.stdout:299.out 2023-11-04T05:31:08.099 INFO:teuthology.orchestra.run.smithi003.stdout:300 2023-11-04T05:31:08.099 INFO:teuthology.orchestra.run.smithi003.stdout:300.out 2023-11-04T05:31:08.099 INFO:teuthology.orchestra.run.smithi003.stdout:301 2023-11-04T05:31:08.099 INFO:teuthology.orchestra.run.smithi003.stdout:301.out 2023-11-04T05:31:08.100 INFO:teuthology.orchestra.run.smithi003.stdout:302 2023-11-04T05:31:08.100 INFO:teuthology.orchestra.run.smithi003.stdout:302.out 2023-11-04T05:31:08.100 INFO:teuthology.orchestra.run.smithi003.stdout:303 2023-11-04T05:31:08.100 INFO:teuthology.orchestra.run.smithi003.stdout:303.out 2023-11-04T05:31:08.100 INFO:teuthology.orchestra.run.smithi003.stdout:304 2023-11-04T05:31:08.101 INFO:teuthology.orchestra.run.smithi003.stdout:304.out 2023-11-04T05:31:08.101 INFO:teuthology.orchestra.run.smithi003.stdout:305 2023-11-04T05:31:08.101 INFO:teuthology.orchestra.run.smithi003.stdout:305.out 2023-11-04T05:31:08.101 INFO:teuthology.orchestra.run.smithi003.stdout:306 2023-11-04T05:31:08.101 INFO:teuthology.orchestra.run.smithi003.stdout:306.out 2023-11-04T05:31:08.102 INFO:teuthology.orchestra.run.smithi003.stdout:307 2023-11-04T05:31:08.102 INFO:teuthology.orchestra.run.smithi003.stdout:307.out 2023-11-04T05:31:08.102 INFO:teuthology.orchestra.run.smithi003.stdout:308 2023-11-04T05:31:08.102 INFO:teuthology.orchestra.run.smithi003.stdout:308.out 2023-11-04T05:31:08.102 INFO:teuthology.orchestra.run.smithi003.stdout:309 2023-11-04T05:31:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:309.out 2023-11-04T05:31:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:310 2023-11-04T05:31:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:310.out 2023-11-04T05:31:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:311 2023-11-04T05:31:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:311.out 2023-11-04T05:31:08.103 INFO:teuthology.orchestra.run.smithi003.stdout:312 2023-11-04T05:31:08.104 INFO:teuthology.orchestra.run.smithi003.stdout:312.out 2023-11-04T05:31:08.104 INFO:teuthology.orchestra.run.smithi003.stdout:313 2023-11-04T05:31:08.104 INFO:teuthology.orchestra.run.smithi003.stdout:313.out 2023-11-04T05:31:08.104 INFO:teuthology.orchestra.run.smithi003.stdout:314 2023-11-04T05:31:08.104 INFO:teuthology.orchestra.run.smithi003.stdout:314.out 2023-11-04T05:31:08.105 INFO:teuthology.orchestra.run.smithi003.stdout:315 2023-11-04T05:31:08.105 INFO:teuthology.orchestra.run.smithi003.stdout:315.out 2023-11-04T05:31:08.105 INFO:teuthology.orchestra.run.smithi003.stdout:316 2023-11-04T05:31:08.105 INFO:teuthology.orchestra.run.smithi003.stdout:316.out 2023-11-04T05:31:08.105 INFO:teuthology.orchestra.run.smithi003.stdout:317 2023-11-04T05:31:08.106 INFO:teuthology.orchestra.run.smithi003.stdout:317.out 2023-11-04T05:31:08.106 INFO:teuthology.orchestra.run.smithi003.stdout:318 2023-11-04T05:31:08.106 INFO:teuthology.orchestra.run.smithi003.stdout:318.out 2023-11-04T05:31:08.106 INFO:teuthology.orchestra.run.smithi003.stdout:319 2023-11-04T05:31:08.106 INFO:teuthology.orchestra.run.smithi003.stdout:319.out 2023-11-04T05:31:08.107 INFO:teuthology.orchestra.run.smithi003.stdout:320 2023-11-04T05:31:08.107 INFO:teuthology.orchestra.run.smithi003.stdout:320.out 2023-11-04T05:31:08.107 INFO:teuthology.orchestra.run.smithi003.stdout:321 2023-11-04T05:31:08.107 INFO:teuthology.orchestra.run.smithi003.stdout:321.out 2023-11-04T05:31:08.107 INFO:teuthology.orchestra.run.smithi003.stdout:322 2023-11-04T05:31:08.108 INFO:teuthology.orchestra.run.smithi003.stdout:322.out 2023-11-04T05:31:08.108 INFO:teuthology.orchestra.run.smithi003.stdout:323 2023-11-04T05:31:08.108 INFO:teuthology.orchestra.run.smithi003.stdout:323.out 2023-11-04T05:31:08.108 INFO:teuthology.orchestra.run.smithi003.stdout:324 2023-11-04T05:31:08.108 INFO:teuthology.orchestra.run.smithi003.stdout:324.out 2023-11-04T05:31:08.109 INFO:teuthology.orchestra.run.smithi003.stdout:325 2023-11-04T05:31:08.109 INFO:teuthology.orchestra.run.smithi003.stdout:325.out 2023-11-04T05:31:08.109 INFO:teuthology.orchestra.run.smithi003.stdout:326 2023-11-04T05:31:08.109 INFO:teuthology.orchestra.run.smithi003.stdout:326.out 2023-11-04T05:31:08.109 INFO:teuthology.orchestra.run.smithi003.stdout:327 2023-11-04T05:31:08.109 INFO:teuthology.orchestra.run.smithi003.stdout:327.out 2023-11-04T05:31:08.110 INFO:teuthology.orchestra.run.smithi003.stdout:328 2023-11-04T05:31:08.110 INFO:teuthology.orchestra.run.smithi003.stdout:328.out 2023-11-04T05:31:08.110 INFO:teuthology.orchestra.run.smithi003.stdout:329 2023-11-04T05:31:08.110 INFO:teuthology.orchestra.run.smithi003.stdout:329.out 2023-11-04T05:31:08.110 INFO:teuthology.orchestra.run.smithi003.stdout:330 2023-11-04T05:31:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:330.out 2023-11-04T05:31:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:331 2023-11-04T05:31:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:331.out 2023-11-04T05:31:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:332 2023-11-04T05:31:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:332.out 2023-11-04T05:31:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:333 2023-11-04T05:31:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:333.out 2023-11-04T05:31:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:334 2023-11-04T05:31:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:334.out 2023-11-04T05:31:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:335 2023-11-04T05:31:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:335.out 2023-11-04T05:31:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:336 2023-11-04T05:31:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:336.out 2023-11-04T05:31:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:337 2023-11-04T05:31:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:337.out 2023-11-04T05:31:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:338 2023-11-04T05:31:08.113 INFO:teuthology.orchestra.run.smithi003.stdout:338.out 2023-11-04T05:31:08.114 INFO:teuthology.orchestra.run.smithi003.stdout:339 2023-11-04T05:31:08.114 INFO:teuthology.orchestra.run.smithi003.stdout:339.out 2023-11-04T05:31:08.114 INFO:teuthology.orchestra.run.smithi003.stdout:340 2023-11-04T05:31:08.114 INFO:teuthology.orchestra.run.smithi003.stdout:340.out 2023-11-04T05:31:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:341 2023-11-04T05:31:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:341.out 2023-11-04T05:31:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:342 2023-11-04T05:31:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:342.out 2023-11-04T05:31:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:343 2023-11-04T05:31:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:343.out 2023-11-04T05:31:08.116 INFO:teuthology.orchestra.run.smithi003.stdout:344 2023-11-04T05:31:08.116 INFO:teuthology.orchestra.run.smithi003.stdout:344.out 2023-11-04T05:31:08.116 INFO:teuthology.orchestra.run.smithi003.stdout:345 2023-11-04T05:31:08.116 INFO:teuthology.orchestra.run.smithi003.stdout:345.out 2023-11-04T05:31:08.116 INFO:teuthology.orchestra.run.smithi003.stdout:346 2023-11-04T05:31:08.117 INFO:teuthology.orchestra.run.smithi003.stdout:346.out 2023-11-04T05:31:08.117 INFO:teuthology.orchestra.run.smithi003.stdout:347 2023-11-04T05:31:08.117 INFO:teuthology.orchestra.run.smithi003.stdout:347.out 2023-11-04T05:31:08.117 INFO:teuthology.orchestra.run.smithi003.stdout:348 2023-11-04T05:31:08.117 INFO:teuthology.orchestra.run.smithi003.stdout:348.out 2023-11-04T05:31:08.118 INFO:teuthology.orchestra.run.smithi003.stdout:349 2023-11-04T05:31:08.118 INFO:teuthology.orchestra.run.smithi003.stdout:349.out 2023-11-04T05:31:08.118 INFO:teuthology.orchestra.run.smithi003.stdout:350 2023-11-04T05:31:08.118 INFO:teuthology.orchestra.run.smithi003.stdout:350.out 2023-11-04T05:31:08.118 INFO:teuthology.orchestra.run.smithi003.stdout:351 2023-11-04T05:31:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:351.out 2023-11-04T05:31:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:352 2023-11-04T05:31:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:352.out 2023-11-04T05:31:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:353 2023-11-04T05:31:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:353.out 2023-11-04T05:31:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:354 2023-11-04T05:31:08.120 INFO:teuthology.orchestra.run.smithi003.stdout:354.out 2023-11-04T05:31:08.120 INFO:teuthology.orchestra.run.smithi003.stdout:355 2023-11-04T05:31:08.120 INFO:teuthology.orchestra.run.smithi003.stdout:355.out 2023-11-04T05:31:08.120 INFO:teuthology.orchestra.run.smithi003.stdout:356 2023-11-04T05:31:08.120 INFO:teuthology.orchestra.run.smithi003.stdout:356.out 2023-11-04T05:31:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:357 2023-11-04T05:31:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:357.out 2023-11-04T05:31:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:358 2023-11-04T05:31:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:358.out 2023-11-04T05:31:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:359 2023-11-04T05:31:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:359.out 2023-11-04T05:31:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:360 2023-11-04T05:31:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:360.out 2023-11-04T05:31:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:361 2023-11-04T05:31:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:361.out 2023-11-04T05:31:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:362 2023-11-04T05:31:08.123 INFO:teuthology.orchestra.run.smithi003.stdout:362.out 2023-11-04T05:31:08.123 INFO:teuthology.orchestra.run.smithi003.stdout:363 2023-11-04T05:31:08.123 INFO:teuthology.orchestra.run.smithi003.stdout:363.out 2023-11-04T05:31:08.123 INFO:teuthology.orchestra.run.smithi003.stdout:364 2023-11-04T05:31:08.123 INFO:teuthology.orchestra.run.smithi003.stdout:364.out 2023-11-04T05:31:08.124 INFO:teuthology.orchestra.run.smithi003.stdout:365 2023-11-04T05:31:08.124 INFO:teuthology.orchestra.run.smithi003.stdout:365.out 2023-11-04T05:31:08.124 INFO:teuthology.orchestra.run.smithi003.stdout:366 2023-11-04T05:31:08.124 INFO:teuthology.orchestra.run.smithi003.stdout:366.out 2023-11-04T05:31:08.124 INFO:teuthology.orchestra.run.smithi003.stdout:367 2023-11-04T05:31:08.125 INFO:teuthology.orchestra.run.smithi003.stdout:367.out 2023-11-04T05:31:08.125 INFO:teuthology.orchestra.run.smithi003.stdout:368 2023-11-04T05:31:08.125 INFO:teuthology.orchestra.run.smithi003.stdout:368.out 2023-11-04T05:31:08.125 INFO:teuthology.orchestra.run.smithi003.stdout:369 2023-11-04T05:31:08.125 INFO:teuthology.orchestra.run.smithi003.stdout:369.out 2023-11-04T05:31:08.125 INFO:teuthology.orchestra.run.smithi003.stdout:370 2023-11-04T05:31:08.126 INFO:teuthology.orchestra.run.smithi003.stdout:370.out 2023-11-04T05:31:08.126 INFO:teuthology.orchestra.run.smithi003.stdout:371 2023-11-04T05:31:08.126 INFO:teuthology.orchestra.run.smithi003.stdout:371.out 2023-11-04T05:31:08.126 INFO:teuthology.orchestra.run.smithi003.stdout:372 2023-11-04T05:31:08.126 INFO:teuthology.orchestra.run.smithi003.stdout:372.out 2023-11-04T05:31:08.127 INFO:teuthology.orchestra.run.smithi003.stdout:373 2023-11-04T05:31:08.127 INFO:teuthology.orchestra.run.smithi003.stdout:373.out 2023-11-04T05:31:08.127 INFO:teuthology.orchestra.run.smithi003.stdout:374 2023-11-04T05:31:08.127 INFO:teuthology.orchestra.run.smithi003.stdout:374.out 2023-11-04T05:31:08.127 INFO:teuthology.orchestra.run.smithi003.stdout:375 2023-11-04T05:31:08.128 INFO:teuthology.orchestra.run.smithi003.stdout:375.out 2023-11-04T05:31:08.128 INFO:teuthology.orchestra.run.smithi003.stdout:376 2023-11-04T05:31:08.128 INFO:teuthology.orchestra.run.smithi003.stdout:376.out 2023-11-04T05:31:08.128 INFO:teuthology.orchestra.run.smithi003.stdout:377 2023-11-04T05:31:08.128 INFO:teuthology.orchestra.run.smithi003.stdout:377.out 2023-11-04T05:31:08.129 INFO:teuthology.orchestra.run.smithi003.stdout:378 2023-11-04T05:31:08.129 INFO:teuthology.orchestra.run.smithi003.stdout:378.out 2023-11-04T05:31:08.129 INFO:teuthology.orchestra.run.smithi003.stdout:379 2023-11-04T05:31:08.129 INFO:teuthology.orchestra.run.smithi003.stdout:379.out 2023-11-04T05:31:08.129 INFO:teuthology.orchestra.run.smithi003.stdout:380 2023-11-04T05:31:08.130 INFO:teuthology.orchestra.run.smithi003.stdout:380.out 2023-11-04T05:31:08.130 INFO:teuthology.orchestra.run.smithi003.stdout:381 2023-11-04T05:31:08.130 INFO:teuthology.orchestra.run.smithi003.stdout:381.out 2023-11-04T05:31:08.130 INFO:teuthology.orchestra.run.smithi003.stdout:382 2023-11-04T05:31:08.130 INFO:teuthology.orchestra.run.smithi003.stdout:382.out 2023-11-04T05:31:08.131 INFO:teuthology.orchestra.run.smithi003.stdout:383 2023-11-04T05:31:08.131 INFO:teuthology.orchestra.run.smithi003.stdout:383.out 2023-11-04T05:31:08.131 INFO:teuthology.orchestra.run.smithi003.stdout:384 2023-11-04T05:31:08.131 INFO:teuthology.orchestra.run.smithi003.stdout:384.out 2023-11-04T05:31:08.131 INFO:teuthology.orchestra.run.smithi003.stdout:385 2023-11-04T05:31:08.132 INFO:teuthology.orchestra.run.smithi003.stdout:385.out 2023-11-04T05:31:08.132 INFO:teuthology.orchestra.run.smithi003.stdout:386 2023-11-04T05:31:08.132 INFO:teuthology.orchestra.run.smithi003.stdout:386.out 2023-11-04T05:31:08.132 INFO:teuthology.orchestra.run.smithi003.stdout:387 2023-11-04T05:31:08.132 INFO:teuthology.orchestra.run.smithi003.stdout:387.out 2023-11-04T05:31:08.133 INFO:teuthology.orchestra.run.smithi003.stdout:388 2023-11-04T05:31:08.133 INFO:teuthology.orchestra.run.smithi003.stdout:388.out 2023-11-04T05:31:08.133 INFO:teuthology.orchestra.run.smithi003.stdout:389 2023-11-04T05:31:08.133 INFO:teuthology.orchestra.run.smithi003.stdout:389.out 2023-11-04T05:31:08.133 INFO:teuthology.orchestra.run.smithi003.stdout:390 2023-11-04T05:31:08.133 INFO:teuthology.orchestra.run.smithi003.stdout:390.out 2023-11-04T05:31:08.134 INFO:teuthology.orchestra.run.smithi003.stdout:391 2023-11-04T05:31:08.134 INFO:teuthology.orchestra.run.smithi003.stdout:391.out 2023-11-04T05:31:08.134 INFO:teuthology.orchestra.run.smithi003.stdout:392 2023-11-04T05:31:08.134 INFO:teuthology.orchestra.run.smithi003.stdout:392.out 2023-11-04T05:31:08.134 INFO:teuthology.orchestra.run.smithi003.stdout:393 2023-11-04T05:31:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:393.out 2023-11-04T05:31:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:394 2023-11-04T05:31:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:394.out 2023-11-04T05:31:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:395 2023-11-04T05:31:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:395.out 2023-11-04T05:31:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:396 2023-11-04T05:31:08.136 INFO:teuthology.orchestra.run.smithi003.stdout:396.out 2023-11-04T05:31:08.136 INFO:teuthology.orchestra.run.smithi003.stdout:397 2023-11-04T05:31:08.136 INFO:teuthology.orchestra.run.smithi003.stdout:397.out 2023-11-04T05:31:08.136 INFO:teuthology.orchestra.run.smithi003.stdout:398 2023-11-04T05:31:08.136 INFO:teuthology.orchestra.run.smithi003.stdout:398.out 2023-11-04T05:31:08.137 INFO:teuthology.orchestra.run.smithi003.stdout:399 2023-11-04T05:31:08.137 INFO:teuthology.orchestra.run.smithi003.stdout:399.out 2023-11-04T05:31:08.137 INFO:teuthology.orchestra.run.smithi003.stdout:400 2023-11-04T05:31:08.137 INFO:teuthology.orchestra.run.smithi003.stdout:400.out 2023-11-04T05:31:08.137 INFO:teuthology.orchestra.run.smithi003.stdout:401 2023-11-04T05:31:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:401.out 2023-11-04T05:31:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:402 2023-11-04T05:31:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:402.out 2023-11-04T05:31:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:403 2023-11-04T05:31:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:403.out 2023-11-04T05:31:08.139 INFO:teuthology.orchestra.run.smithi003.stdout:404 2023-11-04T05:31:08.139 INFO:teuthology.orchestra.run.smithi003.stdout:404.out 2023-11-04T05:31:08.139 INFO:teuthology.orchestra.run.smithi003.stdout:405 2023-11-04T05:31:08.139 INFO:teuthology.orchestra.run.smithi003.stdout:405.out 2023-11-04T05:31:08.139 INFO:teuthology.orchestra.run.smithi003.stdout:406 2023-11-04T05:31:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:406.out 2023-11-04T05:31:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:407 2023-11-04T05:31:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:407.out 2023-11-04T05:31:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:408 2023-11-04T05:31:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:408.out 2023-11-04T05:31:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:409 2023-11-04T05:31:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:409.out 2023-11-04T05:31:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:410 2023-11-04T05:31:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:410.out 2023-11-04T05:31:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:411 2023-11-04T05:31:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:411.out 2023-11-04T05:31:08.142 INFO:teuthology.orchestra.run.smithi003.stdout:412 2023-11-04T05:31:08.142 INFO:teuthology.orchestra.run.smithi003.stdout:412.out 2023-11-04T05:31:08.142 INFO:teuthology.orchestra.run.smithi003.stdout:413 2023-11-04T05:31:08.142 INFO:teuthology.orchestra.run.smithi003.stdout:413.out 2023-11-04T05:31:08.142 INFO:teuthology.orchestra.run.smithi003.stdout:414 2023-11-04T05:31:08.143 INFO:teuthology.orchestra.run.smithi003.stdout:414.out 2023-11-04T05:31:08.143 INFO:teuthology.orchestra.run.smithi003.stdout:415 2023-11-04T05:31:08.143 INFO:teuthology.orchestra.run.smithi003.stdout:415.out 2023-11-04T05:31:08.143 INFO:teuthology.orchestra.run.smithi003.stdout:416 2023-11-04T05:31:08.143 INFO:teuthology.orchestra.run.smithi003.stdout:416.out 2023-11-04T05:31:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:417 2023-11-04T05:31:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:417.out 2023-11-04T05:31:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:418 2023-11-04T05:31:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:418.out 2023-11-04T05:31:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:419 2023-11-04T05:31:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:419.out 2023-11-04T05:31:08.145 INFO:teuthology.orchestra.run.smithi003.stdout:420 2023-11-04T05:31:08.145 INFO:teuthology.orchestra.run.smithi003.stdout:420.out 2023-11-04T05:31:08.145 INFO:teuthology.orchestra.run.smithi003.stdout:421 2023-11-04T05:31:08.145 INFO:teuthology.orchestra.run.smithi003.stdout:421.out 2023-11-04T05:31:08.145 INFO:teuthology.orchestra.run.smithi003.stdout:422 2023-11-04T05:31:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:422.out 2023-11-04T05:31:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:423 2023-11-04T05:31:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:423.out 2023-11-04T05:31:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:424 2023-11-04T05:31:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:424.out 2023-11-04T05:31:08.147 INFO:teuthology.orchestra.run.smithi003.stdout:425 2023-11-04T05:31:08.147 INFO:teuthology.orchestra.run.smithi003.stdout:425.out 2023-11-04T05:31:08.147 INFO:teuthology.orchestra.run.smithi003.stdout:426 2023-11-04T05:31:08.147 INFO:teuthology.orchestra.run.smithi003.stdout:426.out 2023-11-04T05:31:08.147 INFO:teuthology.orchestra.run.smithi003.stdout:427 2023-11-04T05:31:08.147 INFO:teuthology.orchestra.run.smithi003.stdout:427.out 2023-11-04T05:31:08.148 INFO:teuthology.orchestra.run.smithi003.stdout:428 2023-11-04T05:31:08.148 INFO:teuthology.orchestra.run.smithi003.stdout:428.out 2023-11-04T05:31:08.148 INFO:teuthology.orchestra.run.smithi003.stdout:429 2023-11-04T05:31:08.148 INFO:teuthology.orchestra.run.smithi003.stdout:429.out 2023-11-04T05:31:08.148 INFO:teuthology.orchestra.run.smithi003.stdout:430 2023-11-04T05:31:08.149 INFO:teuthology.orchestra.run.smithi003.stdout:430.out 2023-11-04T05:31:08.149 INFO:teuthology.orchestra.run.smithi003.stdout:431 2023-11-04T05:31:08.149 INFO:teuthology.orchestra.run.smithi003.stdout:431.out 2023-11-04T05:31:08.149 INFO:teuthology.orchestra.run.smithi003.stdout:432 2023-11-04T05:31:08.149 INFO:teuthology.orchestra.run.smithi003.stdout:432.out 2023-11-04T05:31:08.150 INFO:teuthology.orchestra.run.smithi003.stdout:433 2023-11-04T05:31:08.150 INFO:teuthology.orchestra.run.smithi003.stdout:433.out 2023-11-04T05:31:08.150 INFO:teuthology.orchestra.run.smithi003.stdout:434 2023-11-04T05:31:08.150 INFO:teuthology.orchestra.run.smithi003.stdout:434.out 2023-11-04T05:31:08.150 INFO:teuthology.orchestra.run.smithi003.stdout:435 2023-11-04T05:31:08.151 INFO:teuthology.orchestra.run.smithi003.stdout:435.out 2023-11-04T05:31:08.151 INFO:teuthology.orchestra.run.smithi003.stdout:436 2023-11-04T05:31:08.151 INFO:teuthology.orchestra.run.smithi003.stdout:436.out 2023-11-04T05:31:08.151 INFO:teuthology.orchestra.run.smithi003.stdout:437 2023-11-04T05:31:08.151 INFO:teuthology.orchestra.run.smithi003.stdout:437.out 2023-11-04T05:31:08.152 INFO:teuthology.orchestra.run.smithi003.stdout:438 2023-11-04T05:31:08.152 INFO:teuthology.orchestra.run.smithi003.stdout:438.out 2023-11-04T05:31:08.152 INFO:teuthology.orchestra.run.smithi003.stdout:439 2023-11-04T05:31:08.152 INFO:teuthology.orchestra.run.smithi003.stdout:439.out 2023-11-04T05:31:08.152 INFO:teuthology.orchestra.run.smithi003.stdout:440 2023-11-04T05:31:08.153 INFO:teuthology.orchestra.run.smithi003.stdout:440.out 2023-11-04T05:31:08.153 INFO:teuthology.orchestra.run.smithi003.stdout:441 2023-11-04T05:31:08.153 INFO:teuthology.orchestra.run.smithi003.stdout:441.out 2023-11-04T05:31:08.153 INFO:teuthology.orchestra.run.smithi003.stdout:442 2023-11-04T05:31:08.153 INFO:teuthology.orchestra.run.smithi003.stdout:442.out 2023-11-04T05:31:08.153 INFO:teuthology.orchestra.run.smithi003.stdout:443 2023-11-04T05:31:08.154 INFO:teuthology.orchestra.run.smithi003.stdout:443.out 2023-11-04T05:31:08.154 INFO:teuthology.orchestra.run.smithi003.stdout:444 2023-11-04T05:31:08.154 INFO:teuthology.orchestra.run.smithi003.stdout:444.out 2023-11-04T05:31:08.154 INFO:teuthology.orchestra.run.smithi003.stdout:445 2023-11-04T05:31:08.154 INFO:teuthology.orchestra.run.smithi003.stdout:445.out 2023-11-04T05:31:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:446 2023-11-04T05:31:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:446.out 2023-11-04T05:31:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:447 2023-11-04T05:31:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:447.out 2023-11-04T05:31:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:448 2023-11-04T05:31:08.156 INFO:teuthology.orchestra.run.smithi003.stdout:448.out 2023-11-04T05:31:08.156 INFO:teuthology.orchestra.run.smithi003.stdout:449 2023-11-04T05:31:08.156 INFO:teuthology.orchestra.run.smithi003.stdout:449.out 2023-11-04T05:31:08.156 INFO:teuthology.orchestra.run.smithi003.stdout:450 2023-11-04T05:31:08.156 INFO:teuthology.orchestra.run.smithi003.stdout:450.out 2023-11-04T05:31:08.157 INFO:teuthology.orchestra.run.smithi003.stdout:451 2023-11-04T05:31:08.157 INFO:teuthology.orchestra.run.smithi003.stdout:451.out 2023-11-04T05:31:08.157 INFO:teuthology.orchestra.run.smithi003.stdout:452 2023-11-04T05:31:08.157 INFO:teuthology.orchestra.run.smithi003.stdout:452.out 2023-11-04T05:31:08.157 INFO:teuthology.orchestra.run.smithi003.stdout:453 2023-11-04T05:31:08.158 INFO:teuthology.orchestra.run.smithi003.stdout:453.out 2023-11-04T05:31:08.158 INFO:teuthology.orchestra.run.smithi003.stdout:454 2023-11-04T05:31:08.158 INFO:teuthology.orchestra.run.smithi003.stdout:454.out 2023-11-04T05:31:08.158 INFO:teuthology.orchestra.run.smithi003.stdout:455 2023-11-04T05:31:08.158 INFO:teuthology.orchestra.run.smithi003.stdout:455.out 2023-11-04T05:31:08.159 INFO:teuthology.orchestra.run.smithi003.stdout:456 2023-11-04T05:31:08.159 INFO:teuthology.orchestra.run.smithi003.stdout:456.out 2023-11-04T05:31:08.159 INFO:teuthology.orchestra.run.smithi003.stdout:457 2023-11-04T05:31:08.159 INFO:teuthology.orchestra.run.smithi003.stdout:457.out 2023-11-04T05:31:08.159 INFO:teuthology.orchestra.run.smithi003.stdout:458 2023-11-04T05:31:08.159 INFO:teuthology.orchestra.run.smithi003.stdout:458.out 2023-11-04T05:31:08.160 INFO:teuthology.orchestra.run.smithi003.stdout:459 2023-11-04T05:31:08.160 INFO:teuthology.orchestra.run.smithi003.stdout:459.out 2023-11-04T05:31:08.160 INFO:teuthology.orchestra.run.smithi003.stdout:460 2023-11-04T05:31:08.160 INFO:teuthology.orchestra.run.smithi003.stdout:460.out 2023-11-04T05:31:08.160 INFO:teuthology.orchestra.run.smithi003.stdout:461 2023-11-04T05:31:08.161 INFO:teuthology.orchestra.run.smithi003.stdout:461.out 2023-11-04T05:31:08.161 INFO:teuthology.orchestra.run.smithi003.stdout:462 2023-11-04T05:31:08.161 INFO:teuthology.orchestra.run.smithi003.stdout:462.out 2023-11-04T05:31:08.161 INFO:teuthology.orchestra.run.smithi003.stdout:463 2023-11-04T05:31:08.161 INFO:teuthology.orchestra.run.smithi003.stdout:463.out 2023-11-04T05:31:08.161 INFO:teuthology.orchestra.run.smithi003.stdout:464 2023-11-04T05:31:08.162 INFO:teuthology.orchestra.run.smithi003.stdout:464.out 2023-11-04T05:31:08.162 INFO:teuthology.orchestra.run.smithi003.stdout:465 2023-11-04T05:31:08.162 INFO:teuthology.orchestra.run.smithi003.stdout:465.out 2023-11-04T05:31:08.162 INFO:teuthology.orchestra.run.smithi003.stdout:466 2023-11-04T05:31:08.162 INFO:teuthology.orchestra.run.smithi003.stdout:466.out 2023-11-04T05:31:08.163 INFO:teuthology.orchestra.run.smithi003.stdout:467 2023-11-04T05:31:08.163 INFO:teuthology.orchestra.run.smithi003.stdout:467.out 2023-11-04T05:31:08.163 INFO:teuthology.orchestra.run.smithi003.stdout:468 2023-11-04T05:31:08.163 INFO:teuthology.orchestra.run.smithi003.stdout:468.out 2023-11-04T05:31:08.163 INFO:teuthology.orchestra.run.smithi003.stdout:469 2023-11-04T05:31:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:469.out 2023-11-04T05:31:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:470 2023-11-04T05:31:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:470.out 2023-11-04T05:31:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:471 2023-11-04T05:31:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:471.out 2023-11-04T05:31:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:472 2023-11-04T05:31:08.165 INFO:teuthology.orchestra.run.smithi003.stdout:472.out 2023-11-04T05:31:08.165 INFO:teuthology.orchestra.run.smithi003.stdout:473 2023-11-04T05:31:08.165 INFO:teuthology.orchestra.run.smithi003.stdout:473.out 2023-11-04T05:31:08.165 INFO:teuthology.orchestra.run.smithi003.stdout:474 2023-11-04T05:31:08.165 INFO:teuthology.orchestra.run.smithi003.stdout:474.out 2023-11-04T05:31:08.166 INFO:teuthology.orchestra.run.smithi003.stdout:475 2023-11-04T05:31:08.166 INFO:teuthology.orchestra.run.smithi003.stdout:475.out 2023-11-04T05:31:08.166 INFO:teuthology.orchestra.run.smithi003.stdout:476 2023-11-04T05:31:08.166 INFO:teuthology.orchestra.run.smithi003.stdout:476.out 2023-11-04T05:31:08.166 INFO:teuthology.orchestra.run.smithi003.stdout:477 2023-11-04T05:31:08.167 INFO:teuthology.orchestra.run.smithi003.stdout:477.out 2023-11-04T05:31:08.167 INFO:teuthology.orchestra.run.smithi003.stdout:478 2023-11-04T05:31:08.167 INFO:teuthology.orchestra.run.smithi003.stdout:478.out 2023-11-04T05:31:08.167 INFO:teuthology.orchestra.run.smithi003.stdout:479 2023-11-04T05:31:08.167 INFO:teuthology.orchestra.run.smithi003.stdout:479.out 2023-11-04T05:31:08.168 INFO:teuthology.orchestra.run.smithi003.stdout:480 2023-11-04T05:31:08.168 INFO:teuthology.orchestra.run.smithi003.stdout:480.out 2023-11-04T05:31:08.168 INFO:teuthology.orchestra.run.smithi003.stdout:481 2023-11-04T05:31:08.168 INFO:teuthology.orchestra.run.smithi003.stdout:481.out 2023-11-04T05:31:08.168 INFO:teuthology.orchestra.run.smithi003.stdout:482 2023-11-04T05:31:08.168 INFO:teuthology.orchestra.run.smithi003.stdout:482.out 2023-11-04T05:31:08.169 INFO:teuthology.orchestra.run.smithi003.stdout:483 2023-11-04T05:31:08.169 INFO:teuthology.orchestra.run.smithi003.stdout:483.out 2023-11-04T05:31:08.169 INFO:teuthology.orchestra.run.smithi003.stdout:484 2023-11-04T05:31:08.169 INFO:teuthology.orchestra.run.smithi003.stdout:484.out 2023-11-04T05:31:08.169 INFO:teuthology.orchestra.run.smithi003.stdout:485 2023-11-04T05:31:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:485.out 2023-11-04T05:31:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:486 2023-11-04T05:31:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:486.out 2023-11-04T05:31:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:487 2023-11-04T05:31:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:487.out 2023-11-04T05:31:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:488 2023-11-04T05:31:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:488.out 2023-11-04T05:31:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:489 2023-11-04T05:31:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:489.out 2023-11-04T05:31:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:490 2023-11-04T05:31:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:490.out 2023-11-04T05:31:08.172 INFO:teuthology.orchestra.run.smithi003.stdout:491 2023-11-04T05:31:08.172 INFO:teuthology.orchestra.run.smithi003.stdout:491.out 2023-11-04T05:31:08.172 INFO:teuthology.orchestra.run.smithi003.stdout:492 2023-11-04T05:31:08.172 INFO:teuthology.orchestra.run.smithi003.stdout:492.out 2023-11-04T05:31:08.172 INFO:teuthology.orchestra.run.smithi003.stdout:493 2023-11-04T05:31:08.173 INFO:teuthology.orchestra.run.smithi003.stdout:493.out 2023-11-04T05:31:08.173 INFO:teuthology.orchestra.run.smithi003.stdout:494 2023-11-04T05:31:08.173 INFO:teuthology.orchestra.run.smithi003.stdout:494.out 2023-11-04T05:31:08.173 INFO:teuthology.orchestra.run.smithi003.stdout:495 2023-11-04T05:31:08.173 INFO:teuthology.orchestra.run.smithi003.stdout:495.out 2023-11-04T05:31:08.174 INFO:teuthology.orchestra.run.smithi003.stdout:496 2023-11-04T05:31:08.174 INFO:teuthology.orchestra.run.smithi003.stdout:496.out 2023-11-04T05:31:08.174 INFO:teuthology.orchestra.run.smithi003.stdout:497 2023-11-04T05:31:08.174 INFO:teuthology.orchestra.run.smithi003.stdout:497.out 2023-11-04T05:31:08.174 INFO:teuthology.orchestra.run.smithi003.stdout:498 2023-11-04T05:31:08.175 INFO:teuthology.orchestra.run.smithi003.stdout:498.out 2023-11-04T05:31:08.175 INFO:teuthology.orchestra.run.smithi003.stdout:499 2023-11-04T05:31:08.175 INFO:teuthology.orchestra.run.smithi003.stdout:499.out 2023-11-04T05:31:08.175 INFO:teuthology.orchestra.run.smithi003.stdout:500 2023-11-04T05:31:08.175 INFO:teuthology.orchestra.run.smithi003.stdout:500.out 2023-11-04T05:31:08.175 INFO:teuthology.orchestra.run.smithi003.stdout:501 2023-11-04T05:31:08.176 INFO:teuthology.orchestra.run.smithi003.stdout:501.out 2023-11-04T05:31:08.176 INFO:teuthology.orchestra.run.smithi003.stdout:502 2023-11-04T05:31:08.176 INFO:teuthology.orchestra.run.smithi003.stdout:502.out 2023-11-04T05:31:08.176 INFO:teuthology.orchestra.run.smithi003.stdout:503 2023-11-04T05:31:08.176 INFO:teuthology.orchestra.run.smithi003.stdout:503.out 2023-11-04T05:31:08.177 INFO:teuthology.orchestra.run.smithi003.stdout:504 2023-11-04T05:31:08.177 INFO:teuthology.orchestra.run.smithi003.stdout:504.out 2023-11-04T05:31:08.177 INFO:teuthology.orchestra.run.smithi003.stdout:505 2023-11-04T05:31:08.177 INFO:teuthology.orchestra.run.smithi003.stdout:505.out 2023-11-04T05:31:08.177 INFO:teuthology.orchestra.run.smithi003.stdout:506 2023-11-04T05:31:08.178 INFO:teuthology.orchestra.run.smithi003.stdout:506.out 2023-11-04T05:31:08.178 INFO:teuthology.orchestra.run.smithi003.stdout:507 2023-11-04T05:31:08.178 INFO:teuthology.orchestra.run.smithi003.stdout:507.out 2023-11-04T05:31:08.178 INFO:teuthology.orchestra.run.smithi003.stdout:508 2023-11-04T05:31:08.178 INFO:teuthology.orchestra.run.smithi003.stdout:508.out 2023-11-04T05:31:08.179 INFO:teuthology.orchestra.run.smithi003.stdout:509 2023-11-04T05:31:08.179 INFO:teuthology.orchestra.run.smithi003.stdout:509.out 2023-11-04T05:31:08.179 INFO:teuthology.orchestra.run.smithi003.stdout:510 2023-11-04T05:31:08.179 INFO:teuthology.orchestra.run.smithi003.stdout:510.out 2023-11-04T05:31:08.179 INFO:teuthology.orchestra.run.smithi003.stdout:511 2023-11-04T05:31:08.179 INFO:teuthology.orchestra.run.smithi003.stdout:511.out 2023-11-04T05:31:08.180 INFO:teuthology.orchestra.run.smithi003.stdout:512 2023-11-04T05:31:08.180 INFO:teuthology.orchestra.run.smithi003.stdout:512.out 2023-11-04T05:31:08.180 INFO:teuthology.orchestra.run.smithi003.stdout:513 2023-11-04T05:31:08.180 INFO:teuthology.orchestra.run.smithi003.stdout:513.out 2023-11-04T05:31:08.180 INFO:teuthology.orchestra.run.smithi003.stdout:514 2023-11-04T05:31:08.181 INFO:teuthology.orchestra.run.smithi003.stdout:514.out 2023-11-04T05:31:08.181 INFO:teuthology.orchestra.run.smithi003.stdout:515 2023-11-04T05:31:08.181 INFO:teuthology.orchestra.run.smithi003.stdout:515.out 2023-11-04T05:31:08.181 INFO:teuthology.orchestra.run.smithi003.stdout:516 2023-11-04T05:31:08.181 INFO:teuthology.orchestra.run.smithi003.stdout:516.out 2023-11-04T05:31:08.182 INFO:teuthology.orchestra.run.smithi003.stdout:517 2023-11-04T05:31:08.182 INFO:teuthology.orchestra.run.smithi003.stdout:517.out 2023-11-04T05:31:08.182 INFO:teuthology.orchestra.run.smithi003.stdout:518 2023-11-04T05:31:08.182 INFO:teuthology.orchestra.run.smithi003.stdout:518.out 2023-11-04T05:31:08.182 INFO:teuthology.orchestra.run.smithi003.stdout:519 2023-11-04T05:31:08.183 INFO:teuthology.orchestra.run.smithi003.stdout:519.out 2023-11-04T05:31:08.183 INFO:teuthology.orchestra.run.smithi003.stdout:520 2023-11-04T05:31:08.183 INFO:teuthology.orchestra.run.smithi003.stdout:520.out 2023-11-04T05:31:08.183 INFO:teuthology.orchestra.run.smithi003.stdout:521 2023-11-04T05:31:08.183 INFO:teuthology.orchestra.run.smithi003.stdout:521.out 2023-11-04T05:31:08.184 INFO:teuthology.orchestra.run.smithi003.stdout:522 2023-11-04T05:31:08.184 INFO:teuthology.orchestra.run.smithi003.stdout:522.out 2023-11-04T05:31:08.184 INFO:teuthology.orchestra.run.smithi003.stdout:523 2023-11-04T05:31:08.184 INFO:teuthology.orchestra.run.smithi003.stdout:523.out 2023-11-04T05:31:08.184 INFO:teuthology.orchestra.run.smithi003.stdout:524 2023-11-04T05:31:08.184 INFO:teuthology.orchestra.run.smithi003.stdout:524.out 2023-11-04T05:31:08.185 INFO:teuthology.orchestra.run.smithi003.stdout:525 2023-11-04T05:31:08.185 INFO:teuthology.orchestra.run.smithi003.stdout:525.out 2023-11-04T05:31:08.185 INFO:teuthology.orchestra.run.smithi003.stdout:526 2023-11-04T05:31:08.185 INFO:teuthology.orchestra.run.smithi003.stdout:526.out 2023-11-04T05:31:08.185 INFO:teuthology.orchestra.run.smithi003.stdout:527 2023-11-04T05:31:08.186 INFO:teuthology.orchestra.run.smithi003.stdout:527.out 2023-11-04T05:31:08.186 INFO:teuthology.orchestra.run.smithi003.stdout:528 2023-11-04T05:31:08.186 INFO:teuthology.orchestra.run.smithi003.stdout:528.out 2023-11-04T05:31:08.186 INFO:teuthology.orchestra.run.smithi003.stdout:529 2023-11-04T05:31:08.186 INFO:teuthology.orchestra.run.smithi003.stdout:529.out 2023-11-04T05:31:08.187 INFO:teuthology.orchestra.run.smithi003.stdout:530 2023-11-04T05:31:08.187 INFO:teuthology.orchestra.run.smithi003.stdout:530.out 2023-11-04T05:31:08.187 INFO:teuthology.orchestra.run.smithi003.stdout:531 2023-11-04T05:31:08.187 INFO:teuthology.orchestra.run.smithi003.stdout:531.out 2023-11-04T05:31:08.187 INFO:teuthology.orchestra.run.smithi003.stdout:532 2023-11-04T05:31:08.187 INFO:teuthology.orchestra.run.smithi003.stdout:532.out 2023-11-04T05:31:08.188 INFO:teuthology.orchestra.run.smithi003.stdout:533 2023-11-04T05:31:08.188 INFO:teuthology.orchestra.run.smithi003.stdout:533.out 2023-11-04T05:31:08.188 INFO:teuthology.orchestra.run.smithi003.stdout:534 2023-11-04T05:31:08.188 INFO:teuthology.orchestra.run.smithi003.stdout:534.out 2023-11-04T05:31:08.188 INFO:teuthology.orchestra.run.smithi003.stdout:535 2023-11-04T05:31:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:535.out 2023-11-04T05:31:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:536 2023-11-04T05:31:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:536.out 2023-11-04T05:31:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:537 2023-11-04T05:31:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:537.out 2023-11-04T05:31:08.190 INFO:teuthology.orchestra.run.smithi003.stdout:538 2023-11-04T05:31:08.190 INFO:teuthology.orchestra.run.smithi003.stdout:538.out 2023-11-04T05:31:08.190 INFO:teuthology.orchestra.run.smithi003.stdout:539 2023-11-04T05:31:08.190 INFO:teuthology.orchestra.run.smithi003.stdout:539.out 2023-11-04T05:31:08.190 INFO:teuthology.orchestra.run.smithi003.stdout:540 2023-11-04T05:31:08.191 INFO:teuthology.orchestra.run.smithi003.stdout:540.out 2023-11-04T05:31:08.191 INFO:teuthology.orchestra.run.smithi003.stdout:541 2023-11-04T05:31:08.191 INFO:teuthology.orchestra.run.smithi003.stdout:541.out 2023-11-04T05:31:08.191 INFO:teuthology.orchestra.run.smithi003.stdout:542 2023-11-04T05:31:08.191 INFO:teuthology.orchestra.run.smithi003.stdout:542.out 2023-11-04T05:31:08.192 INFO:teuthology.orchestra.run.smithi003.stdout:543 2023-11-04T05:31:08.192 INFO:teuthology.orchestra.run.smithi003.stdout:543.out 2023-11-04T05:31:08.192 INFO:teuthology.orchestra.run.smithi003.stdout:544 2023-11-04T05:31:08.192 INFO:teuthology.orchestra.run.smithi003.stdout:544.out 2023-11-04T05:31:08.192 INFO:teuthology.orchestra.run.smithi003.stdout:545 2023-11-04T05:31:08.193 INFO:teuthology.orchestra.run.smithi003.stdout:545.out 2023-11-04T05:31:08.193 INFO:teuthology.orchestra.run.smithi003.stdout:546 2023-11-04T05:31:08.193 INFO:teuthology.orchestra.run.smithi003.stdout:546.out 2023-11-04T05:31:08.193 INFO:teuthology.orchestra.run.smithi003.stdout:547 2023-11-04T05:31:08.193 INFO:teuthology.orchestra.run.smithi003.stdout:547.out 2023-11-04T05:31:08.194 INFO:teuthology.orchestra.run.smithi003.stdout:548 2023-11-04T05:31:08.194 INFO:teuthology.orchestra.run.smithi003.stdout:548.out 2023-11-04T05:31:08.194 INFO:teuthology.orchestra.run.smithi003.stdout:549 2023-11-04T05:31:08.194 INFO:teuthology.orchestra.run.smithi003.stdout:549.out 2023-11-04T05:31:08.195 INFO:teuthology.orchestra.run.smithi003.stdout:550 2023-11-04T05:31:08.195 INFO:teuthology.orchestra.run.smithi003.stdout:550.out 2023-11-04T05:31:08.195 INFO:teuthology.orchestra.run.smithi003.stdout:551 2023-11-04T05:31:08.195 INFO:teuthology.orchestra.run.smithi003.stdout:551.out 2023-11-04T05:31:08.195 INFO:teuthology.orchestra.run.smithi003.stdout:552 2023-11-04T05:31:08.195 INFO:teuthology.orchestra.run.smithi003.stdout:552.out 2023-11-04T05:31:08.196 INFO:teuthology.orchestra.run.smithi003.stdout:553 2023-11-04T05:31:08.196 INFO:teuthology.orchestra.run.smithi003.stdout:553.out 2023-11-04T05:31:08.196 INFO:teuthology.orchestra.run.smithi003.stdout:554 2023-11-04T05:31:08.196 INFO:teuthology.orchestra.run.smithi003.stdout:554.out 2023-11-04T05:31:08.196 INFO:teuthology.orchestra.run.smithi003.stdout:555 2023-11-04T05:31:08.197 INFO:teuthology.orchestra.run.smithi003.stdout:555.out 2023-11-04T05:31:08.197 INFO:teuthology.orchestra.run.smithi003.stdout:556 2023-11-04T05:31:08.197 INFO:teuthology.orchestra.run.smithi003.stdout:556.out 2023-11-04T05:31:08.197 INFO:teuthology.orchestra.run.smithi003.stdout:557 2023-11-04T05:31:08.197 INFO:teuthology.orchestra.run.smithi003.stdout:557.out 2023-11-04T05:31:08.198 INFO:teuthology.orchestra.run.smithi003.stdout:558 2023-11-04T05:31:08.198 INFO:teuthology.orchestra.run.smithi003.stdout:558.out 2023-11-04T05:31:08.198 INFO:teuthology.orchestra.run.smithi003.stdout:559 2023-11-04T05:31:08.198 INFO:teuthology.orchestra.run.smithi003.stdout:559.out 2023-11-04T05:31:08.198 INFO:teuthology.orchestra.run.smithi003.stdout:560 2023-11-04T05:31:08.199 INFO:teuthology.orchestra.run.smithi003.stdout:560.out 2023-11-04T05:31:08.199 INFO:teuthology.orchestra.run.smithi003.stdout:561 2023-11-04T05:31:08.199 INFO:teuthology.orchestra.run.smithi003.stdout:561.out 2023-11-04T05:31:08.199 INFO:teuthology.orchestra.run.smithi003.stdout:562 2023-11-04T05:31:08.199 INFO:teuthology.orchestra.run.smithi003.stdout:562.out 2023-11-04T05:31:08.200 INFO:teuthology.orchestra.run.smithi003.stdout:563 2023-11-04T05:31:08.200 INFO:teuthology.orchestra.run.smithi003.stdout:563.out 2023-11-04T05:31:08.200 INFO:teuthology.orchestra.run.smithi003.stdout:564 2023-11-04T05:31:08.200 INFO:teuthology.orchestra.run.smithi003.stdout:564.out 2023-11-04T05:31:08.200 INFO:teuthology.orchestra.run.smithi003.stdout:565 2023-11-04T05:31:08.201 INFO:teuthology.orchestra.run.smithi003.stdout:565.out 2023-11-04T05:31:08.201 INFO:teuthology.orchestra.run.smithi003.stdout:566 2023-11-04T05:31:08.201 INFO:teuthology.orchestra.run.smithi003.stdout:566.out 2023-11-04T05:31:08.201 INFO:teuthology.orchestra.run.smithi003.stdout:567 2023-11-04T05:31:08.201 INFO:teuthology.orchestra.run.smithi003.stdout:567.out 2023-11-04T05:31:08.202 INFO:teuthology.orchestra.run.smithi003.stdout:568 2023-11-04T05:31:08.202 INFO:teuthology.orchestra.run.smithi003.stdout:568.out 2023-11-04T05:31:08.202 INFO:teuthology.orchestra.run.smithi003.stdout:569 2023-11-04T05:31:08.202 INFO:teuthology.orchestra.run.smithi003.stdout:569.out 2023-11-04T05:31:08.202 INFO:teuthology.orchestra.run.smithi003.stdout:570 2023-11-04T05:31:08.203 INFO:teuthology.orchestra.run.smithi003.stdout:570.out 2023-11-04T05:31:08.203 INFO:teuthology.orchestra.run.smithi003.stdout:571 2023-11-04T05:31:08.203 INFO:teuthology.orchestra.run.smithi003.stdout:571.out 2023-11-04T05:31:08.203 INFO:teuthology.orchestra.run.smithi003.stdout:572 2023-11-04T05:31:08.203 INFO:teuthology.orchestra.run.smithi003.stdout:572.out 2023-11-04T05:31:08.204 INFO:teuthology.orchestra.run.smithi003.stdout:573 2023-11-04T05:31:08.204 INFO:teuthology.orchestra.run.smithi003.stdout:573.out 2023-11-04T05:31:08.204 INFO:teuthology.orchestra.run.smithi003.stdout:574 2023-11-04T05:31:08.204 INFO:teuthology.orchestra.run.smithi003.stdout:574.out 2023-11-04T05:31:08.204 INFO:teuthology.orchestra.run.smithi003.stdout:575 2023-11-04T05:31:08.204 INFO:teuthology.orchestra.run.smithi003.stdout:575.out 2023-11-04T05:31:08.205 INFO:teuthology.orchestra.run.smithi003.stdout:576 2023-11-04T05:31:08.205 INFO:teuthology.orchestra.run.smithi003.stdout:576.out 2023-11-04T05:31:08.205 INFO:teuthology.orchestra.run.smithi003.stdout:577 2023-11-04T05:31:08.205 INFO:teuthology.orchestra.run.smithi003.stdout:577.out 2023-11-04T05:31:08.205 INFO:teuthology.orchestra.run.smithi003.stdout:578 2023-11-04T05:31:08.206 INFO:teuthology.orchestra.run.smithi003.stdout:578.out 2023-11-04T05:31:08.206 INFO:teuthology.orchestra.run.smithi003.stdout:579 2023-11-04T05:31:08.206 INFO:teuthology.orchestra.run.smithi003.stdout:579.out 2023-11-04T05:31:08.206 INFO:teuthology.orchestra.run.smithi003.stdout:580 2023-11-04T05:31:08.206 INFO:teuthology.orchestra.run.smithi003.stdout:580.out 2023-11-04T05:31:08.206 INFO:teuthology.orchestra.run.smithi003.stdout:581 2023-11-04T05:31:08.207 INFO:teuthology.orchestra.run.smithi003.stdout:581.out 2023-11-04T05:31:08.207 INFO:teuthology.orchestra.run.smithi003.stdout:582 2023-11-04T05:31:08.207 INFO:teuthology.orchestra.run.smithi003.stdout:582.out 2023-11-04T05:31:08.207 INFO:teuthology.orchestra.run.smithi003.stdout:583 2023-11-04T05:31:08.207 INFO:teuthology.orchestra.run.smithi003.stdout:583.out 2023-11-04T05:31:08.208 INFO:teuthology.orchestra.run.smithi003.stdout:584 2023-11-04T05:31:08.208 INFO:teuthology.orchestra.run.smithi003.stdout:584.out 2023-11-04T05:31:08.208 INFO:teuthology.orchestra.run.smithi003.stdout:585 2023-11-04T05:31:08.208 INFO:teuthology.orchestra.run.smithi003.stdout:585.out 2023-11-04T05:31:08.208 INFO:teuthology.orchestra.run.smithi003.stdout:586 2023-11-04T05:31:08.209 INFO:teuthology.orchestra.run.smithi003.stdout:586.out 2023-11-04T05:31:08.209 INFO:teuthology.orchestra.run.smithi003.stdout:587 2023-11-04T05:31:08.209 INFO:teuthology.orchestra.run.smithi003.stdout:587.out 2023-11-04T05:31:08.209 INFO:teuthology.orchestra.run.smithi003.stdout:588 2023-11-04T05:31:08.209 INFO:teuthology.orchestra.run.smithi003.stdout:588.out 2023-11-04T05:31:08.210 INFO:teuthology.orchestra.run.smithi003.stdout:589 2023-11-04T05:31:08.210 INFO:teuthology.orchestra.run.smithi003.stdout:589.out 2023-11-04T05:31:08.210 INFO:teuthology.orchestra.run.smithi003.stdout:590 2023-11-04T05:31:08.210 INFO:teuthology.orchestra.run.smithi003.stdout:590.out 2023-11-04T05:31:08.210 INFO:teuthology.orchestra.run.smithi003.stdout:591 2023-11-04T05:31:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:591.out 2023-11-04T05:31:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:592 2023-11-04T05:31:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:592.out 2023-11-04T05:31:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:593 2023-11-04T05:31:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:593.out 2023-11-04T05:31:08.211 INFO:teuthology.orchestra.run.smithi003.stdout:594 2023-11-04T05:31:08.212 INFO:teuthology.orchestra.run.smithi003.stdout:594.out 2023-11-04T05:31:08.212 INFO:teuthology.orchestra.run.smithi003.stdout:595 2023-11-04T05:31:08.212 INFO:teuthology.orchestra.run.smithi003.stdout:595.out 2023-11-04T05:31:08.212 INFO:teuthology.orchestra.run.smithi003.stdout:596 2023-11-04T05:31:08.212 INFO:teuthology.orchestra.run.smithi003.stdout:596.out 2023-11-04T05:31:08.213 INFO:teuthology.orchestra.run.smithi003.stdout:597 2023-11-04T05:31:08.213 INFO:teuthology.orchestra.run.smithi003.stdout:597.out 2023-11-04T05:31:08.213 INFO:teuthology.orchestra.run.smithi003.stdout:598 2023-11-04T05:31:08.213 INFO:teuthology.orchestra.run.smithi003.stdout:598.out 2023-11-04T05:31:08.213 INFO:teuthology.orchestra.run.smithi003.stdout:599 2023-11-04T05:31:08.214 INFO:teuthology.orchestra.run.smithi003.stdout:599.out 2023-11-04T05:31:08.214 INFO:teuthology.orchestra.run.smithi003.stdout:600 2023-11-04T05:31:08.214 INFO:teuthology.orchestra.run.smithi003.stdout:600.out 2023-11-04T05:31:08.214 INFO:teuthology.orchestra.run.smithi003.stdout:601 2023-11-04T05:31:08.214 INFO:teuthology.orchestra.run.smithi003.stdout:601.out 2023-11-04T05:31:08.215 INFO:teuthology.orchestra.run.smithi003.stdout:602 2023-11-04T05:31:08.215 INFO:teuthology.orchestra.run.smithi003.stdout:602.out 2023-11-04T05:31:08.215 INFO:teuthology.orchestra.run.smithi003.stdout:603 2023-11-04T05:31:08.215 INFO:teuthology.orchestra.run.smithi003.stdout:603.out 2023-11-04T05:31:08.215 INFO:teuthology.orchestra.run.smithi003.stdout:604 2023-11-04T05:31:08.216 INFO:teuthology.orchestra.run.smithi003.stdout:604.out 2023-11-04T05:31:08.216 INFO:teuthology.orchestra.run.smithi003.stdout:605 2023-11-04T05:31:08.216 INFO:teuthology.orchestra.run.smithi003.stdout:605.out 2023-11-04T05:31:08.216 INFO:teuthology.orchestra.run.smithi003.stdout:606 2023-11-04T05:31:08.216 INFO:teuthology.orchestra.run.smithi003.stdout:606.out 2023-11-04T05:31:08.216 INFO:teuthology.orchestra.run.smithi003.stdout:607 2023-11-04T05:31:08.217 INFO:teuthology.orchestra.run.smithi003.stdout:607.out 2023-11-04T05:31:08.217 INFO:teuthology.orchestra.run.smithi003.stdout:608 2023-11-04T05:31:08.217 INFO:teuthology.orchestra.run.smithi003.stdout:608.out 2023-11-04T05:31:08.217 INFO:teuthology.orchestra.run.smithi003.stdout:609 2023-11-04T05:31:08.217 INFO:teuthology.orchestra.run.smithi003.stdout:609.out 2023-11-04T05:31:08.218 INFO:teuthology.orchestra.run.smithi003.stdout:610 2023-11-04T05:31:08.218 INFO:teuthology.orchestra.run.smithi003.stdout:610.out 2023-11-04T05:31:08.218 INFO:teuthology.orchestra.run.smithi003.stdout:611 2023-11-04T05:31:08.218 INFO:teuthology.orchestra.run.smithi003.stdout:611.out 2023-11-04T05:31:08.218 INFO:teuthology.orchestra.run.smithi003.stdout:612 2023-11-04T05:31:08.219 INFO:teuthology.orchestra.run.smithi003.stdout:612.out 2023-11-04T05:31:08.219 INFO:teuthology.orchestra.run.smithi003.stdout:613 2023-11-04T05:31:08.219 INFO:teuthology.orchestra.run.smithi003.stdout:613.out 2023-11-04T05:31:08.219 INFO:teuthology.orchestra.run.smithi003.stdout:614 2023-11-04T05:31:08.219 INFO:teuthology.orchestra.run.smithi003.stdout:614.out 2023-11-04T05:31:08.220 INFO:teuthology.orchestra.run.smithi003.stdout:615 2023-11-04T05:31:08.220 INFO:teuthology.orchestra.run.smithi003.stdout:615.out 2023-11-04T05:31:08.220 INFO:teuthology.orchestra.run.smithi003.stdout:616 2023-11-04T05:31:08.220 INFO:teuthology.orchestra.run.smithi003.stdout:616.out 2023-11-04T05:31:08.220 INFO:teuthology.orchestra.run.smithi003.stdout:617 2023-11-04T05:31:08.220 INFO:teuthology.orchestra.run.smithi003.stdout:617.out 2023-11-04T05:31:08.221 INFO:teuthology.orchestra.run.smithi003.stdout:618 2023-11-04T05:31:08.221 INFO:teuthology.orchestra.run.smithi003.stdout:618.out 2023-11-04T05:31:08.221 INFO:teuthology.orchestra.run.smithi003.stdout:619 2023-11-04T05:31:08.221 INFO:teuthology.orchestra.run.smithi003.stdout:619.out 2023-11-04T05:31:08.221 INFO:teuthology.orchestra.run.smithi003.stdout:620 2023-11-04T05:31:08.222 INFO:teuthology.orchestra.run.smithi003.stdout:620.out 2023-11-04T05:31:08.222 INFO:teuthology.orchestra.run.smithi003.stdout:621 2023-11-04T05:31:08.222 INFO:teuthology.orchestra.run.smithi003.stdout:621.out 2023-11-04T05:31:08.222 INFO:teuthology.orchestra.run.smithi003.stdout:622 2023-11-04T05:31:08.222 INFO:teuthology.orchestra.run.smithi003.stdout:622.out 2023-11-04T05:31:08.223 INFO:teuthology.orchestra.run.smithi003.stdout:623 2023-11-04T05:31:08.223 INFO:teuthology.orchestra.run.smithi003.stdout:623.out 2023-11-04T05:31:08.223 INFO:teuthology.orchestra.run.smithi003.stdout:624 2023-11-04T05:31:08.223 INFO:teuthology.orchestra.run.smithi003.stdout:624.out 2023-11-04T05:31:08.223 INFO:teuthology.orchestra.run.smithi003.stdout:625 2023-11-04T05:31:08.223 INFO:teuthology.orchestra.run.smithi003.stdout:625.out 2023-11-04T05:31:08.224 INFO:teuthology.orchestra.run.smithi003.stdout:626 2023-11-04T05:31:08.224 INFO:teuthology.orchestra.run.smithi003.stdout:626.out 2023-11-04T05:31:08.224 INFO:teuthology.orchestra.run.smithi003.stdout:627 2023-11-04T05:31:08.224 INFO:teuthology.orchestra.run.smithi003.stdout:627.out 2023-11-04T05:31:08.224 INFO:teuthology.orchestra.run.smithi003.stdout:628 2023-11-04T05:31:08.225 INFO:teuthology.orchestra.run.smithi003.stdout:628.out 2023-11-04T05:31:08.225 INFO:teuthology.orchestra.run.smithi003.stdout:629 2023-11-04T05:31:08.225 INFO:teuthology.orchestra.run.smithi003.stdout:629.out 2023-11-04T05:31:08.225 INFO:teuthology.orchestra.run.smithi003.stdout:630 2023-11-04T05:31:08.225 INFO:teuthology.orchestra.run.smithi003.stdout:630.out 2023-11-04T05:31:08.226 INFO:teuthology.orchestra.run.smithi003.stdout:631 2023-11-04T05:31:08.226 INFO:teuthology.orchestra.run.smithi003.stdout:631.out 2023-11-04T05:31:08.226 INFO:teuthology.orchestra.run.smithi003.stdout:632 2023-11-04T05:31:08.226 INFO:teuthology.orchestra.run.smithi003.stdout:632.out 2023-11-04T05:31:08.226 INFO:teuthology.orchestra.run.smithi003.stdout:633 2023-11-04T05:31:08.226 INFO:teuthology.orchestra.run.smithi003.stdout:633.out 2023-11-04T05:31:08.227 INFO:teuthology.orchestra.run.smithi003.stdout:634 2023-11-04T05:31:08.227 INFO:teuthology.orchestra.run.smithi003.stdout:634.out 2023-11-04T05:31:08.227 INFO:teuthology.orchestra.run.smithi003.stdout:635 2023-11-04T05:31:08.227 INFO:teuthology.orchestra.run.smithi003.stdout:635.out 2023-11-04T05:31:08.227 INFO:teuthology.orchestra.run.smithi003.stdout:636 2023-11-04T05:31:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:636.out 2023-11-04T05:31:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:637 2023-11-04T05:31:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:637.out 2023-11-04T05:31:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:638 2023-11-04T05:31:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:638.out 2023-11-04T05:31:08.228 INFO:teuthology.orchestra.run.smithi003.stdout:639 2023-11-04T05:31:08.229 INFO:teuthology.orchestra.run.smithi003.stdout:639.out 2023-11-04T05:31:08.229 INFO:teuthology.orchestra.run.smithi003.stdout:640 2023-11-04T05:31:08.229 INFO:teuthology.orchestra.run.smithi003.stdout:640.out 2023-11-04T05:31:08.229 INFO:teuthology.orchestra.run.smithi003.stdout:641 2023-11-04T05:31:08.229 INFO:teuthology.orchestra.run.smithi003.stdout:641.out 2023-11-04T05:31:08.230 INFO:teuthology.orchestra.run.smithi003.stdout:642 2023-11-04T05:31:08.230 INFO:teuthology.orchestra.run.smithi003.stdout:642.out 2023-11-04T05:31:08.230 INFO:teuthology.orchestra.run.smithi003.stdout:643 2023-11-04T05:31:08.230 INFO:teuthology.orchestra.run.smithi003.stdout:643.out 2023-11-04T05:31:08.230 INFO:teuthology.orchestra.run.smithi003.stdout:644 2023-11-04T05:31:08.231 INFO:teuthology.orchestra.run.smithi003.stdout:644.out 2023-11-04T05:31:08.231 INFO:teuthology.orchestra.run.smithi003.stdout:645 2023-11-04T05:31:08.231 INFO:teuthology.orchestra.run.smithi003.stdout:645.out 2023-11-04T05:31:08.231 INFO:teuthology.orchestra.run.smithi003.stdout:646 2023-11-04T05:31:08.231 INFO:teuthology.orchestra.run.smithi003.stdout:646.out 2023-11-04T05:31:08.232 INFO:teuthology.orchestra.run.smithi003.stdout:647 2023-11-04T05:31:08.232 INFO:teuthology.orchestra.run.smithi003.stdout:647.out 2023-11-04T05:31:08.232 INFO:teuthology.orchestra.run.smithi003.stdout:648 2023-11-04T05:31:08.232 INFO:teuthology.orchestra.run.smithi003.stdout:648.out 2023-11-04T05:31:08.232 INFO:teuthology.orchestra.run.smithi003.stdout:649 2023-11-04T05:31:08.233 INFO:teuthology.orchestra.run.smithi003.stdout:649.out 2023-11-04T05:31:08.233 INFO:teuthology.orchestra.run.smithi003.stdout:650 2023-11-04T05:31:08.233 INFO:teuthology.orchestra.run.smithi003.stdout:650.out 2023-11-04T05:31:08.233 INFO:teuthology.orchestra.run.smithi003.stdout:651 2023-11-04T05:31:08.233 INFO:teuthology.orchestra.run.smithi003.stdout:651.out 2023-11-04T05:31:08.234 INFO:teuthology.orchestra.run.smithi003.stdout:652 2023-11-04T05:31:08.234 INFO:teuthology.orchestra.run.smithi003.stdout:652.out 2023-11-04T05:31:08.234 INFO:teuthology.orchestra.run.smithi003.stdout:653 2023-11-04T05:31:08.234 INFO:teuthology.orchestra.run.smithi003.stdout:653.out 2023-11-04T05:31:08.234 INFO:teuthology.orchestra.run.smithi003.stdout:654 2023-11-04T05:31:08.235 INFO:teuthology.orchestra.run.smithi003.stdout:654.out 2023-11-04T05:31:08.235 INFO:teuthology.orchestra.run.smithi003.stdout:655 2023-11-04T05:31:08.235 INFO:teuthology.orchestra.run.smithi003.stdout:655.out 2023-11-04T05:31:08.235 INFO:teuthology.orchestra.run.smithi003.stdout:656 2023-11-04T05:31:08.235 INFO:teuthology.orchestra.run.smithi003.stdout:656.out 2023-11-04T05:31:08.235 INFO:teuthology.orchestra.run.smithi003.stdout:657 2023-11-04T05:31:08.236 INFO:teuthology.orchestra.run.smithi003.stdout:657.out 2023-11-04T05:31:08.236 INFO:teuthology.orchestra.run.smithi003.stdout:658 2023-11-04T05:31:08.236 INFO:teuthology.orchestra.run.smithi003.stdout:658.out 2023-11-04T05:31:08.236 INFO:teuthology.orchestra.run.smithi003.stdout:659 2023-11-04T05:31:08.236 INFO:teuthology.orchestra.run.smithi003.stdout:659.out 2023-11-04T05:31:08.237 INFO:teuthology.orchestra.run.smithi003.stdout:660 2023-11-04T05:31:08.237 INFO:teuthology.orchestra.run.smithi003.stdout:660.out 2023-11-04T05:31:08.237 INFO:teuthology.orchestra.run.smithi003.stdout:661 2023-11-04T05:31:08.237 INFO:teuthology.orchestra.run.smithi003.stdout:661.out 2023-11-04T05:31:08.237 INFO:teuthology.orchestra.run.smithi003.stdout:662 2023-11-04T05:31:08.238 INFO:teuthology.orchestra.run.smithi003.stdout:662.out 2023-11-04T05:31:08.238 INFO:teuthology.orchestra.run.smithi003.stdout:663 2023-11-04T05:31:08.238 INFO:teuthology.orchestra.run.smithi003.stdout:663.out 2023-11-04T05:31:08.238 INFO:teuthology.orchestra.run.smithi003.stdout:664 2023-11-04T05:31:08.238 INFO:teuthology.orchestra.run.smithi003.stdout:664.out 2023-11-04T05:31:08.239 INFO:teuthology.orchestra.run.smithi003.stdout:665 2023-11-04T05:31:08.239 INFO:teuthology.orchestra.run.smithi003.stdout:665.out 2023-11-04T05:31:08.239 INFO:teuthology.orchestra.run.smithi003.stdout:666 2023-11-04T05:31:08.239 INFO:teuthology.orchestra.run.smithi003.stdout:666.out 2023-11-04T05:31:08.239 INFO:teuthology.orchestra.run.smithi003.stdout:667 2023-11-04T05:31:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:667.out 2023-11-04T05:31:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:668 2023-11-04T05:31:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:668.out 2023-11-04T05:31:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:669 2023-11-04T05:31:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:669.out 2023-11-04T05:31:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:670 2023-11-04T05:31:08.241 INFO:teuthology.orchestra.run.smithi003.stdout:670.out 2023-11-04T05:31:08.241 INFO:teuthology.orchestra.run.smithi003.stdout:671 2023-11-04T05:31:08.241 INFO:teuthology.orchestra.run.smithi003.stdout:671.out 2023-11-04T05:31:08.241 INFO:teuthology.orchestra.run.smithi003.stdout:672 2023-11-04T05:31:08.241 INFO:teuthology.orchestra.run.smithi003.stdout:672.out 2023-11-04T05:31:08.242 INFO:teuthology.orchestra.run.smithi003.stdout:673 2023-11-04T05:31:08.242 INFO:teuthology.orchestra.run.smithi003.stdout:673.out 2023-11-04T05:31:08.242 INFO:teuthology.orchestra.run.smithi003.stdout:674 2023-11-04T05:31:08.242 INFO:teuthology.orchestra.run.smithi003.stdout:674.out 2023-11-04T05:31:08.242 INFO:teuthology.orchestra.run.smithi003.stdout:675 2023-11-04T05:31:08.243 INFO:teuthology.orchestra.run.smithi003.stdout:675.out 2023-11-04T05:31:08.243 INFO:teuthology.orchestra.run.smithi003.stdout:676 2023-11-04T05:31:08.243 INFO:teuthology.orchestra.run.smithi003.stdout:676.out 2023-11-04T05:31:08.243 INFO:teuthology.orchestra.run.smithi003.stdout:677 2023-11-04T05:31:08.243 INFO:teuthology.orchestra.run.smithi003.stdout:677.out 2023-11-04T05:31:08.244 INFO:teuthology.orchestra.run.smithi003.stdout:678 2023-11-04T05:31:08.244 INFO:teuthology.orchestra.run.smithi003.stdout:678.out 2023-11-04T05:31:08.244 INFO:teuthology.orchestra.run.smithi003.stdout:679 2023-11-04T05:31:08.244 INFO:teuthology.orchestra.run.smithi003.stdout:679.out 2023-11-04T05:31:08.244 INFO:teuthology.orchestra.run.smithi003.stdout:680 2023-11-04T05:31:08.244 INFO:teuthology.orchestra.run.smithi003.stdout:680.out 2023-11-04T05:31:08.245 INFO:teuthology.orchestra.run.smithi003.stdout:681 2023-11-04T05:31:08.245 INFO:teuthology.orchestra.run.smithi003.stdout:681.out 2023-11-04T05:31:08.245 INFO:teuthology.orchestra.run.smithi003.stdout:682 2023-11-04T05:31:08.245 INFO:teuthology.orchestra.run.smithi003.stdout:682.out 2023-11-04T05:31:08.245 INFO:teuthology.orchestra.run.smithi003.stdout:683 2023-11-04T05:31:08.246 INFO:teuthology.orchestra.run.smithi003.stdout:683.out 2023-11-04T05:31:08.246 INFO:teuthology.orchestra.run.smithi003.stdout:684 2023-11-04T05:31:08.246 INFO:teuthology.orchestra.run.smithi003.stdout:684.out 2023-11-04T05:31:08.246 INFO:teuthology.orchestra.run.smithi003.stdout:685 2023-11-04T05:31:08.246 INFO:teuthology.orchestra.run.smithi003.stdout:685.out 2023-11-04T05:31:08.247 INFO:teuthology.orchestra.run.smithi003.stdout:686 2023-11-04T05:31:08.247 INFO:teuthology.orchestra.run.smithi003.stdout:686.out 2023-11-04T05:31:08.247 INFO:teuthology.orchestra.run.smithi003.stdout:687 2023-11-04T05:31:08.247 INFO:teuthology.orchestra.run.smithi003.stdout:687.out 2023-11-04T05:31:08.247 INFO:teuthology.orchestra.run.smithi003.stdout:688 2023-11-04T05:31:08.247 INFO:teuthology.orchestra.run.smithi003.stdout:688.out 2023-11-04T05:31:08.248 INFO:teuthology.orchestra.run.smithi003.stdout:689 2023-11-04T05:31:08.248 INFO:teuthology.orchestra.run.smithi003.stdout:689.out 2023-11-04T05:31:08.248 INFO:teuthology.orchestra.run.smithi003.stdout:690 2023-11-04T05:31:08.248 INFO:teuthology.orchestra.run.smithi003.stdout:690.out 2023-11-04T05:31:08.248 INFO:teuthology.orchestra.run.smithi003.stdout:691 2023-11-04T05:31:08.249 INFO:teuthology.orchestra.run.smithi003.stdout:691.out 2023-11-04T05:31:08.249 INFO:teuthology.orchestra.run.smithi003.stdout:692 2023-11-04T05:31:08.249 INFO:teuthology.orchestra.run.smithi003.stdout:692.out 2023-11-04T05:31:08.249 INFO:teuthology.orchestra.run.smithi003.stdout:693 2023-11-04T05:31:08.249 INFO:teuthology.orchestra.run.smithi003.stdout:693.out 2023-11-04T05:31:08.249 INFO:teuthology.orchestra.run.smithi003.stdout:694 2023-11-04T05:31:08.250 INFO:teuthology.orchestra.run.smithi003.stdout:694.out 2023-11-04T05:31:08.250 INFO:teuthology.orchestra.run.smithi003.stdout:695 2023-11-04T05:31:08.250 INFO:teuthology.orchestra.run.smithi003.stdout:695.out 2023-11-04T05:31:08.250 INFO:teuthology.orchestra.run.smithi003.stdout:696 2023-11-04T05:31:08.250 INFO:teuthology.orchestra.run.smithi003.stdout:696.out 2023-11-04T05:31:08.251 INFO:teuthology.orchestra.run.smithi003.stdout:697 2023-11-04T05:31:08.251 INFO:teuthology.orchestra.run.smithi003.stdout:697.out 2023-11-04T05:31:08.251 INFO:teuthology.orchestra.run.smithi003.stdout:698 2023-11-04T05:31:08.251 INFO:teuthology.orchestra.run.smithi003.stdout:698.out 2023-11-04T05:31:08.251 INFO:teuthology.orchestra.run.smithi003.stdout:699 2023-11-04T05:31:08.251 INFO:teuthology.orchestra.run.smithi003.stdout:699.out 2023-11-04T05:31:08.252 INFO:teuthology.orchestra.run.smithi003.stdout:700 2023-11-04T05:31:08.252 INFO:teuthology.orchestra.run.smithi003.stdout:700.out 2023-11-04T05:31:08.252 INFO:teuthology.orchestra.run.smithi003.stdout:701 2023-11-04T05:31:08.252 INFO:teuthology.orchestra.run.smithi003.stdout:701.out 2023-11-04T05:31:08.252 INFO:teuthology.orchestra.run.smithi003.stdout:702 2023-11-04T05:31:08.253 INFO:teuthology.orchestra.run.smithi003.stdout:702.out 2023-11-04T05:31:08.253 INFO:teuthology.orchestra.run.smithi003.stdout:703 2023-11-04T05:31:08.253 INFO:teuthology.orchestra.run.smithi003.stdout:703.out 2023-11-04T05:31:08.253 INFO:teuthology.orchestra.run.smithi003.stdout:704 2023-11-04T05:31:08.253 INFO:teuthology.orchestra.run.smithi003.stdout:704.out 2023-11-04T05:31:08.254 INFO:teuthology.orchestra.run.smithi003.stdout:705 2023-11-04T05:31:08.254 INFO:teuthology.orchestra.run.smithi003.stdout:705.out 2023-11-04T05:31:08.254 INFO:teuthology.orchestra.run.smithi003.stdout:706 2023-11-04T05:31:08.254 INFO:teuthology.orchestra.run.smithi003.stdout:706.out 2023-11-04T05:31:08.254 INFO:teuthology.orchestra.run.smithi003.stdout:707 2023-11-04T05:31:08.255 INFO:teuthology.orchestra.run.smithi003.stdout:707.out 2023-11-04T05:31:08.255 INFO:teuthology.orchestra.run.smithi003.stdout:708 2023-11-04T05:31:08.255 INFO:teuthology.orchestra.run.smithi003.stdout:708.out 2023-11-04T05:31:08.255 INFO:teuthology.orchestra.run.smithi003.stdout:709 2023-11-04T05:31:08.255 INFO:teuthology.orchestra.run.smithi003.stdout:709.out 2023-11-04T05:31:08.255 INFO:teuthology.orchestra.run.smithi003.stdout:710 2023-11-04T05:31:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:710.out 2023-11-04T05:31:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:711 2023-11-04T05:31:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:711.out 2023-11-04T05:31:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:712 2023-11-04T05:31:08.256 INFO:teuthology.orchestra.run.smithi003.stdout:712.out 2023-11-04T05:31:08.257 INFO:teuthology.orchestra.run.smithi003.stdout:713 2023-11-04T05:31:08.257 INFO:teuthology.orchestra.run.smithi003.stdout:713.out 2023-11-04T05:31:08.257 INFO:teuthology.orchestra.run.smithi003.stdout:714 2023-11-04T05:31:08.257 INFO:teuthology.orchestra.run.smithi003.stdout:714.out 2023-11-04T05:31:08.257 INFO:teuthology.orchestra.run.smithi003.stdout:715 2023-11-04T05:31:08.258 INFO:teuthology.orchestra.run.smithi003.stdout:715.out 2023-11-04T05:31:08.258 INFO:teuthology.orchestra.run.smithi003.stdout:716 2023-11-04T05:31:08.258 INFO:teuthology.orchestra.run.smithi003.stdout:716.out 2023-11-04T05:31:08.258 INFO:teuthology.orchestra.run.smithi003.stdout:717 2023-11-04T05:31:08.258 INFO:teuthology.orchestra.run.smithi003.stdout:717.out 2023-11-04T05:31:08.259 INFO:teuthology.orchestra.run.smithi003.stdout:718 2023-11-04T05:31:08.259 INFO:teuthology.orchestra.run.smithi003.stdout:718.out 2023-11-04T05:31:08.259 INFO:teuthology.orchestra.run.smithi003.stdout:719 2023-11-04T05:31:08.259 INFO:teuthology.orchestra.run.smithi003.stdout:719.out 2023-11-04T05:31:08.259 INFO:teuthology.orchestra.run.smithi003.stdout:720 2023-11-04T05:31:08.260 INFO:teuthology.orchestra.run.smithi003.stdout:720.out 2023-11-04T05:31:08.260 INFO:teuthology.orchestra.run.smithi003.stdout:721 2023-11-04T05:31:08.260 INFO:teuthology.orchestra.run.smithi003.stdout:721.out 2023-11-04T05:31:08.260 INFO:teuthology.orchestra.run.smithi003.stdout:722 2023-11-04T05:31:08.260 INFO:teuthology.orchestra.run.smithi003.stdout:722.out 2023-11-04T05:31:08.260 INFO:teuthology.orchestra.run.smithi003.stdout:723 2023-11-04T05:31:08.261 INFO:teuthology.orchestra.run.smithi003.stdout:723.out 2023-11-04T05:31:08.261 INFO:teuthology.orchestra.run.smithi003.stdout:724 2023-11-04T05:31:08.261 INFO:teuthology.orchestra.run.smithi003.stdout:724.out 2023-11-04T05:31:08.261 INFO:teuthology.orchestra.run.smithi003.stdout:725 2023-11-04T05:31:08.261 INFO:teuthology.orchestra.run.smithi003.stdout:725.out 2023-11-04T05:31:08.262 INFO:teuthology.orchestra.run.smithi003.stdout:726 2023-11-04T05:31:08.262 INFO:teuthology.orchestra.run.smithi003.stdout:726.out 2023-11-04T05:31:08.262 INFO:teuthology.orchestra.run.smithi003.stdout:727 2023-11-04T05:31:08.262 INFO:teuthology.orchestra.run.smithi003.stdout:727.out 2023-11-04T05:31:08.262 INFO:teuthology.orchestra.run.smithi003.stdout:728 2023-11-04T05:31:08.263 INFO:teuthology.orchestra.run.smithi003.stdout:728.out 2023-11-04T05:31:08.263 INFO:teuthology.orchestra.run.smithi003.stdout:729 2023-11-04T05:31:08.263 INFO:teuthology.orchestra.run.smithi003.stdout:729.out 2023-11-04T05:31:08.263 INFO:teuthology.orchestra.run.smithi003.stdout:730 2023-11-04T05:31:08.263 INFO:teuthology.orchestra.run.smithi003.stdout:730.out 2023-11-04T05:31:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:731 2023-11-04T05:31:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:731.out 2023-11-04T05:31:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:732 2023-11-04T05:31:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:732.out 2023-11-04T05:31:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:group.list 2023-11-04T05:31:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:Makefile 2023-11-04T05:31:08.265 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-11-04T05:31:08.507 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:08.511 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:08.512 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:08.513 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:08.513 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:13.120 INFO:teuthology.orchestra.run.smithi003.stdout:generic/001 5s 2023-11-04T05:31:13.124 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/001 2023-11-04T05:31:13.124 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:31:13.126 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:13.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:31:13.860 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-11-04T05:31:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:14.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:14.727 INFO:teuthology.orchestra.run.smithi003.stdout:generic/002 0s 2023-11-04T05:31:14.732 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/002 2023-11-04T05:31:14.732 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:31:14.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:14.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:31:14.803 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-11-04T05:31:15.026 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:15.030 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:15.031 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:15.031 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:15.032 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:15.371 INFO:teuthology.orchestra.run.smithi003.stdout:generic/003 [not run] atime not maintained by ceph 2023-11-04T05:31:15.374 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/003 2023-11-04T05:31:15.375 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/003 2023-11-04T05:31:15.375 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:31:15.377 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:15.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:31:15.400 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-11-04T05:31:15.617 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:15.622 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:15.622 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:15.623 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:15.623 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:15.956 INFO:teuthology.orchestra.run.smithi003.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-11-04T05:31:15.960 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/004 2023-11-04T05:31:15.960 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/004 2023-11-04T05:31:15.960 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:31:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:15.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:31:15.991 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-11-04T05:31:16.209 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:16.215 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:16.215 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:16.215 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:16.692 INFO:teuthology.orchestra.run.smithi003.stdout:generic/005 0s 2023-11-04T05:31:16.697 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/005 2023-11-04T05:31:16.697 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:31:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:16.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:31:16.779 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-11-04T05:31:17.016 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:17.020 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:17.021 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:17.022 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:21.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:31:21.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:31:40.966 INFO:teuthology.orchestra.run.smithi003.stdout:generic/006 23s 2023-11-04T05:31:41.049 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/006 2023-11-04T05:31:41.050 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:31:41.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:41.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:31:41.181 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-11-04T05:31:41.383 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:31:41.387 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:31:41.388 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:31:41.389 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:31:41.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:31:51.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:31:51.970 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:32:22.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:32:22.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:32:30.304 INFO:teuthology.orchestra.run.smithi003.stdout:generic/007 49s 2023-11-04T05:32:30.427 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/007 2023-11-04T05:32:30.428 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:32:30.429 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:30.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:32:30.538 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-11-04T05:32:30.761 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:32:30.765 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:32:30.765 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:32:30.766 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:32:30.766 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:31.163 INFO:teuthology.orchestra.run.smithi003.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-11-04T05:32:31.166 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/008 2023-11-04T05:32:31.167 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/008 2023-11-04T05:32:31.167 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:32:31.168 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:31.190 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:32:31.191 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-11-04T05:32:31.451 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:32:31.455 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:32:31.456 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:32:31.457 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:32:31.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:31.861 INFO:teuthology.orchestra.run.smithi003.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-11-04T05:32:31.865 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/009 2023-11-04T05:32:31.865 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/009 2023-11-04T05:32:31.865 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:32:31.867 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:31.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:32:31.895 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-11-04T05:32:32.114 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:32:32.118 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:32:32.118 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:32:32.119 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:32:32.119 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:32.444 INFO:teuthology.orchestra.run.smithi003.stdout:generic/010 [not run] /tmp/tmp.ByRml5cCmzxfstests-dev/src/dbtest not built 2023-11-04T05:32:32.449 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/010 2023-11-04T05:32:32.449 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/010 2023-11-04T05:32:32.449 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:32:32.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:32.478 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:32:32.479 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-11-04T05:32:32.700 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:32:32.704 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:32:32.704 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:32:32.705 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:32:32.706 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:32:52.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:32:52.376 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:33:03.067 INFO:teuthology.orchestra.run.smithi003.stdout:generic/011 31s 2023-11-04T05:33:03.123 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/011 2023-11-04T05:33:03.123 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:33:03.125 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:33:03.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:33:03.507 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-11-04T05:33:03.779 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:33:03.797 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:33:03.798 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:33:03.801 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:33:03.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:33:04.312 INFO:teuthology.orchestra.run.smithi003.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:33:04.316 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/012 2023-11-04T05:33:04.316 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/012 2023-11-04T05:33:04.316 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:33:04.318 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:33:04.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:33:04.404 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-11-04T05:33:04.630 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:33:04.634 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:33:04.634 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:33:04.635 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:33:04.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:33:22.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:33:22.710 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:33:53.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:33:53.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:34:02.037 INFO:teuthology.orchestra.run.smithi003.stdout:generic/013 58s 2023-11-04T05:34:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/013 2023-11-04T05:34:02.044 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:34:02.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:34:02.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:34:02.087 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-11-04T05:34:02.443 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:34:02.450 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:34:02.450 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:34:02.452 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:34:02.452 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:34:23.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:34:23.290 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:34:53.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:34:53.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:35:23.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:35:23.946 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:35:54.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:35:54.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:36:24.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:36:24.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:36:54.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:36:54.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:37:24.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:37:24.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:37:54.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:37:54.762 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:38:24.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:38:24.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:38:36.207 INFO:teuthology.orchestra.run.smithi003.stdout:generic/014 274s 2023-11-04T05:38:36.215 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/014 2023-11-04T05:38:36.215 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:36.218 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:36.265 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:36.266 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-11-04T05:38:36.616 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:36.620 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:36.621 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:36.622 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:36.622 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:37.080 INFO:teuthology.orchestra.run.smithi003.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T05:38:37.086 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/015 2023-11-04T05:38:37.087 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/015 2023-11-04T05:38:37.087 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:37.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:37.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:37.116 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-11-04T05:38:37.525 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:37.533 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:37.533 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:37.535 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:37.535 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:38.148 INFO:teuthology.orchestra.run.smithi003.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:38:38.153 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/016 2023-11-04T05:38:38.153 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/016 2023-11-04T05:38:38.153 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:38.157 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:38.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:38.439 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-11-04T05:38:38.872 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:38.882 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:38.882 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:38.883 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:38.884 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:39.370 INFO:teuthology.orchestra.run.smithi003.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T05:38:39.383 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/017 2023-11-04T05:38:39.383 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/017 2023-11-04T05:38:39.383 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:39.385 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:39.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:39.427 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-11-04T05:38:39.756 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:39.764 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:39.764 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:39.765 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:39.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2023-11-04T05:38:40.452 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/018 2023-11-04T05:38:40.452 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/018 2023-11-04T05:38:40.452 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:40.454 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:40.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:40.486 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-11-04T05:38:40.837 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:40.849 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:40.850 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:40.853 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:40.853 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:41.314 INFO:teuthology.orchestra.run.smithi003.stdout:generic/019 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:38:41.324 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/019 2023-11-04T05:38:41.324 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/019 2023-11-04T05:38:41.324 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:41.327 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:41.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:41.351 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-11-04T05:38:41.707 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:41.720 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:41.723 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:41.724 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:41.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:45.143 INFO:teuthology.orchestra.run.smithi003.stdout:generic/020 4s 2023-11-04T05:38:45.149 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/020 2023-11-04T05:38:45.149 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:45.151 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:45.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:45.395 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-11-04T05:38:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:45.824 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:45.825 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:45.825 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:46.429 INFO:teuthology.orchestra.run.smithi003.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:38:46.435 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/021 2023-11-04T05:38:46.435 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/021 2023-11-04T05:38:46.435 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:46.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:46.694 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-11-04T05:38:47.119 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:47.139 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:47.139 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:47.140 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:47.140 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:47.774 INFO:teuthology.orchestra.run.smithi003.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:38:47.782 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/022 2023-11-04T05:38:47.782 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/022 2023-11-04T05:38:47.782 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:47.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:48.112 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:48.113 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-11-04T05:38:48.517 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:48.526 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:48.526 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:48.527 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:48.527 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:50.352 INFO:teuthology.orchestra.run.smithi003.stdout:generic/023 2s 2023-11-04T05:38:50.358 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/023 2023-11-04T05:38:50.359 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:50.361 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:50.717 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:50.718 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-11-04T05:38:51.103 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:51.108 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:51.109 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:51.110 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:51.625 INFO:teuthology.orchestra.run.smithi003.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2023-11-04T05:38:51.631 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/024 2023-11-04T05:38:51.631 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/024 2023-11-04T05:38:51.631 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:51.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:51.955 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-11-04T05:38:52.367 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:52.376 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:52.382 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:52.384 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:52.884 INFO:teuthology.orchestra.run.smithi003.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2023-11-04T05:38:52.890 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/025 2023-11-04T05:38:52.890 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/025 2023-11-04T05:38:52.890 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:53.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:53.115 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-11-04T05:38:53.550 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:53.557 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:53.560 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:53.561 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:53.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:54.119 INFO:teuthology.orchestra.run.smithi003.stdout:generic/026 [not run] ceph does not define maximum ACL count 2023-11-04T05:38:54.126 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/026 2023-11-04T05:38:54.126 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/026 2023-11-04T05:38:54.127 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:54.128 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:54.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:54.170 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-11-04T05:38:54.509 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:54.520 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:54.521 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:54.522 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:54.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:54.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:38:54.969 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:38:54.985 INFO:teuthology.orchestra.run.smithi003.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T05:38:54.990 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/027 2023-11-04T05:38:54.990 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/027 2023-11-04T05:38:54.991 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:38:54.993 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:38:55.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:38:55.029 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-11-04T05:38:55.413 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:38:55.417 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:38:55.418 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:38:55.419 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:38:55.419 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:00.918 INFO:teuthology.orchestra.run.smithi003.stdout:generic/028 5s 2023-11-04T05:39:00.921 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/028 2023-11-04T05:39:00.922 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:00.924 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:00.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:00.964 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-11-04T05:39:01.329 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:01.334 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:01.335 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:01.336 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:01.337 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:05.954 INFO:teuthology.orchestra.run.smithi003.stdout:generic/029 4s 2023-11-04T05:39:05.961 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/029 2023-11-04T05:39:05.961 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:05.966 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:05.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:05.978 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-11-04T05:39:06.328 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:06.339 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:06.340 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:06.341 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:06.341 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:10.984 INFO:teuthology.orchestra.run.smithi003.stdout:generic/030 4s 2023-11-04T05:39:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/030 2023-11-04T05:39:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:10.994 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:11.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:11.010 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-11-04T05:39:11.349 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:11.355 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:11.356 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:11.357 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:11.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:11.951 INFO:teuthology.orchestra.run.smithi003.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-11-04T05:39:11.956 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/031 2023-11-04T05:39:11.956 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/031 2023-11-04T05:39:11.956 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:11.960 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:11.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:11.995 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-11-04T05:39:12.302 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:12.313 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:12.316 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:12.317 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:12.317 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:12.809 INFO:teuthology.orchestra.run.smithi003.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:39:12.815 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/032 2023-11-04T05:39:12.815 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/032 2023-11-04T05:39:12.815 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:12.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:12.857 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:12.858 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-11-04T05:39:13.232 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:13.236 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:13.238 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:13.242 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:13.242 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-11-04T05:39:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/033 2023-11-04T05:39:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/033 2023-11-04T05:39:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:13.776 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:13.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:13.815 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-11-04T05:39:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:14.176 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:14.683 INFO:teuthology.orchestra.run.smithi003.stdout:generic/034 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:14.688 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/034 2023-11-04T05:39:14.688 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/034 2023-11-04T05:39:14.688 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:14.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:14.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:14.714 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-11-04T05:39:15.074 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:15.081 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:15.082 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:15.083 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:15.083 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:15.640 INFO:teuthology.orchestra.run.smithi003.stdout:generic/035 0s 2023-11-04T05:39:15.646 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/035 2023-11-04T05:39:15.646 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:15.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:15.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:15.987 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-11-04T05:39:16.355 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:16.362 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:16.362 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:16.363 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:25.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:39:25.057 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:39:26.941 INFO:teuthology.orchestra.run.smithi003.stdout:generic/036 10s 2023-11-04T05:39:26.947 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/036 2023-11-04T05:39:26.947 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:26.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:27.276 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:27.277 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-11-04T05:39:27.650 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:27.654 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:27.654 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:27.656 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:27.656 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:33.915 INFO:teuthology.orchestra.run.smithi003.stdout:generic/037 6s 2023-11-04T05:39:33.982 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/037 2023-11-04T05:39:33.983 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:33.986 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:33.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:33.998 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-11-04T05:39:34.370 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:34.384 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:34.384 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:34.385 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:34.385 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:34.891 INFO:teuthology.orchestra.run.smithi003.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:39:34.897 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/038 2023-11-04T05:39:34.897 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/038 2023-11-04T05:39:34.897 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:34.912 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:34.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:34.962 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-11-04T05:39:35.330 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:35.339 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:35.339 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:35.340 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:35.340 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:35.821 INFO:teuthology.orchestra.run.smithi003.stdout:generic/039 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:35.824 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/039 2023-11-04T05:39:35.825 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/039 2023-11-04T05:39:35.825 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:35.827 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:35.880 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:35.880 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-11-04T05:39:36.248 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:36.254 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:36.255 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:36.256 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:36.256 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:36.781 INFO:teuthology.orchestra.run.smithi003.stdout:generic/040 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:36.791 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/040 2023-11-04T05:39:36.791 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/040 2023-11-04T05:39:36.791 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:36.799 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:36.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:36.832 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-11-04T05:39:37.153 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:37.164 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:37.165 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:37.166 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:37.166 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:37.691 INFO:teuthology.orchestra.run.smithi003.stdout:generic/041 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/041 2023-11-04T05:39:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/041 2023-11-04T05:39:37.697 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:37.701 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:37.739 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:37.740 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-11-04T05:39:38.052 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:38.061 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:38.062 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:38.063 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:38.063 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:38.743 INFO:teuthology.orchestra.run.smithi003.stdout:generic/042 [not run] ceph does not support shutdown 2023-11-04T05:39:38.773 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/042 2023-11-04T05:39:38.773 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/042 2023-11-04T05:39:38.774 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:38.776 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:38.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:38.811 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-11-04T05:39:39.174 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:39.180 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:39.180 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:39.182 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:39.182 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:39.863 INFO:teuthology.orchestra.run.smithi003.stdout:generic/043 [not run] ceph does not support shutdown 2023-11-04T05:39:39.883 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/043 2023-11-04T05:39:39.883 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/043 2023-11-04T05:39:39.883 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:39.888 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:39.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:39.917 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-11-04T05:39:40.299 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:40.305 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:40.305 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:40.306 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:40.306 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:41.005 INFO:teuthology.orchestra.run.smithi003.stdout:generic/044 [not run] ceph does not support shutdown 2023-11-04T05:39:41.026 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/044 2023-11-04T05:39:41.026 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/044 2023-11-04T05:39:41.026 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:41.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:41.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:41.069 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-11-04T05:39:41.437 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:41.446 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:41.447 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:41.448 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:41.448 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:42.138 INFO:teuthology.orchestra.run.smithi003.stdout:generic/045 [not run] ceph does not support shutdown 2023-11-04T05:39:42.154 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/045 2023-11-04T05:39:42.155 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/045 2023-11-04T05:39:42.155 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:42.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:42.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:42.181 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-11-04T05:39:42.543 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:42.551 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:42.551 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:42.554 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:42.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:43.316 INFO:teuthology.orchestra.run.smithi003.stdout:generic/046 [not run] ceph does not support shutdown 2023-11-04T05:39:43.336 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/046 2023-11-04T05:39:43.336 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/046 2023-11-04T05:39:43.337 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:43.341 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:43.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:43.385 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-11-04T05:39:43.707 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:43.712 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:43.712 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:43.714 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:43.714 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:44.348 INFO:teuthology.orchestra.run.smithi003.stdout:generic/047 [not run] ceph does not support shutdown 2023-11-04T05:39:44.373 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/047 2023-11-04T05:39:44.373 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/047 2023-11-04T05:39:44.373 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:44.375 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:44.402 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:44.402 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-11-04T05:39:44.749 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:44.762 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:44.763 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:44.764 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:44.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:generic/048 [not run] ceph does not support shutdown 2023-11-04T05:39:45.512 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/048 2023-11-04T05:39:45.512 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/048 2023-11-04T05:39:45.512 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:45.514 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:45.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:45.541 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-11-04T05:39:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:45.889 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:46.546 INFO:teuthology.orchestra.run.smithi003.stdout:generic/049 [not run] ceph does not support shutdown 2023-11-04T05:39:46.560 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/049 2023-11-04T05:39:46.560 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/049 2023-11-04T05:39:46.560 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:46.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:46.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:46.596 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-11-04T05:39:46.962 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:46.973 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:46.974 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:46.975 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:46.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:47.591 INFO:teuthology.orchestra.run.smithi003.stdout:generic/050 [not run] ceph does not support shutdown 2023-11-04T05:39:47.617 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/050 2023-11-04T05:39:47.618 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/050 2023-11-04T05:39:47.618 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:47.623 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:47.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:47.656 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-11-04T05:39:48.009 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:48.014 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:48.017 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:48.020 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:48.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:48.731 INFO:teuthology.orchestra.run.smithi003.stdout:generic/051 [not run] ceph does not support shutdown 2023-11-04T05:39:48.737 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/051 2023-11-04T05:39:48.737 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/051 2023-11-04T05:39:48.737 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:48.739 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:48.768 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:48.769 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-11-04T05:39:49.107 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:49.115 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:49.115 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:49.116 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:49.117 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:49.794 INFO:teuthology.orchestra.run.smithi003.stdout:generic/052 [not run] ceph does not support shutdown 2023-11-04T05:39:49.815 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/052 2023-11-04T05:39:49.816 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/052 2023-11-04T05:39:49.816 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:49.819 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:49.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:49.844 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-11-04T05:39:50.177 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:50.184 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:50.185 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:50.186 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:50.186 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:51.375 INFO:teuthology.orchestra.run.smithi003.stdout:generic/053 1s 2023-11-04T05:39:51.436 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/053 2023-11-04T05:39:51.437 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:51.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:51.466 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-11-04T05:39:51.778 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:51.783 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:51.784 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:51.785 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:52.497 INFO:teuthology.orchestra.run.smithi003.stdout:generic/054 [not run] ceph does not support shutdown 2023-11-04T05:39:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/054 2023-11-04T05:39:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/054 2023-11-04T05:39:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:52.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:52.558 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-11-04T05:39:52.914 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:52.920 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:52.922 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:52.922 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:53.611 INFO:teuthology.orchestra.run.smithi003.stdout:generic/055 [not run] ceph does not support shutdown 2023-11-04T05:39:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/055 2023-11-04T05:39:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/055 2023-11-04T05:39:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:53.639 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:53.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:53.661 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-11-04T05:39:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:54.019 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:54.019 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:54.021 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:54.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:54.536 INFO:teuthology.orchestra.run.smithi003.stdout:generic/056 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:54.542 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/056 2023-11-04T05:39:54.542 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/056 2023-11-04T05:39:54.542 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:54.544 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:54.581 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:54.581 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-11-04T05:39:54.929 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:54.933 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:54.934 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:54.935 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:54.935 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:55.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:39:55.153 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:39:55.489 INFO:teuthology.orchestra.run.smithi003.stdout:generic/057 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:55.492 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/057 2023-11-04T05:39:55.492 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/057 2023-11-04T05:39:55.492 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:55.495 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:55.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:55.525 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-11-04T05:39:55.870 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:55.877 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:55.878 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:55.882 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:55.883 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:56.470 INFO:teuthology.orchestra.run.smithi003.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:39:56.475 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/058 2023-11-04T05:39:56.475 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/058 2023-11-04T05:39:56.475 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:56.477 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:56.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:56.626 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-11-04T05:39:57.161 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:57.165 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:57.166 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:57.167 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:57.167 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:57.711 INFO:teuthology.orchestra.run.smithi003.stdout:generic/059 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:39:57.716 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/059 2023-11-04T05:39:57.717 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/059 2023-11-04T05:39:57.717 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:57.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:57.759 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-11-04T05:39:58.110 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:58.119 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:58.119 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:58.120 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:58.809 INFO:teuthology.orchestra.run.smithi003.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:39:58.822 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/060 2023-11-04T05:39:58.822 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/060 2023-11-04T05:39:58.822 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:59.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:39:59.211 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-11-04T05:39:59.458 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:39:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:39:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:39:59.463 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:39:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:39:59.891 INFO:teuthology.orchestra.run.smithi003.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:39:59.895 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/061 2023-11-04T05:39:59.896 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/061 2023-11-04T05:39:59.896 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:39:59.898 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:00.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:00.199 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-11-04T05:40:00.454 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:00.458 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:00.459 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:00.460 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:00.460 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:02.288 INFO:teuthology.orchestra.run.smithi003.stdout:generic/062 2s 2023-11-04T05:40:02.296 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/062 2023-11-04T05:40:02.296 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:02.298 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:02.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:02.332 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-11-04T05:40:02.560 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:02.564 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:02.564 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:02.565 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:02.565 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:03.030 INFO:teuthology.orchestra.run.smithi003.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:40:03.034 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/063 2023-11-04T05:40:03.035 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/063 2023-11-04T05:40:03.035 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:03.037 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:03.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:03.256 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-11-04T05:40:03.589 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:03.593 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:03.594 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:03.595 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:03.595 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:03.933 INFO:teuthology.orchestra.run.smithi003.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T05:40:03.937 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/064 2023-11-04T05:40:03.937 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/064 2023-11-04T05:40:03.937 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:03.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:03.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:03.968 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-11-04T05:40:04.186 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:04.190 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:04.190 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:04.192 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:04.192 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:04.548 INFO:teuthology.orchestra.run.smithi003.stdout:generic/065 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:40:04.553 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/065 2023-11-04T05:40:04.553 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/065 2023-11-04T05:40:04.553 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:04.555 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:04.580 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:04.581 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-11-04T05:40:04.796 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:04.800 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:04.800 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:04.801 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:04.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:05.145 INFO:teuthology.orchestra.run.smithi003.stdout:generic/066 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:40:05.150 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/066 2023-11-04T05:40:05.150 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/066 2023-11-04T05:40:05.150 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:05.152 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:05.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:05.181 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-11-04T05:40:05.408 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:05.412 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:05.413 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:05.413 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:05.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:05.837 INFO:teuthology.orchestra.run.smithi003.stdout:generic/067 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:40:05.841 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/067 2023-11-04T05:40:05.841 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/067 2023-11-04T05:40:05.841 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:05.843 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:05.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:05.878 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-11-04T05:40:06.103 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:06.107 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:06.107 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:06.108 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:06.108 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:06.589 INFO:teuthology.orchestra.run.smithi003.stdout:generic/068 [not run] ceph does not support freezing 2023-11-04T05:40:06.593 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/068 2023-11-04T05:40:06.593 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/068 2023-11-04T05:40:06.594 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:06.595 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:06.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:06.615 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-11-04T05:40:06.837 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:06.841 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:06.841 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:06.842 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:06.843 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:13.628 INFO:teuthology.orchestra.run.smithi003.stdout:generic/069 6s 2023-11-04T05:40:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/069 2023-11-04T05:40:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:13.664 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:13.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:13.704 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-11-04T05:40:13.931 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:13.935 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:13.936 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:13.937 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:13.937 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:25.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:40:25.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:40:41.791 INFO:teuthology.orchestra.run.smithi003.stdout:generic/070 27s 2023-11-04T05:40:41.796 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/070 2023-11-04T05:40:41.796 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:41.798 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:41.890 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:41.890 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-11-04T05:40:42.366 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:42.378 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:42.379 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:42.379 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:42.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:42.920 INFO:teuthology.orchestra.run.smithi003.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T05:40:42.924 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/071 2023-11-04T05:40:42.925 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/071 2023-11-04T05:40:42.925 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:42.930 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:42.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:42.967 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-11-04T05:40:43.350 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:43.352 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:43.354 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:43.355 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:43.355 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:43.942 INFO:teuthology.orchestra.run.smithi003.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-11-04T05:40:43.950 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/072 2023-11-04T05:40:43.951 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/072 2023-11-04T05:40:43.951 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:43.952 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:43.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:43.997 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-11-04T05:40:44.350 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:44.360 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:44.360 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:44.363 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:44.363 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:44.852 INFO:teuthology.orchestra.run.smithi003.stdout:generic/073 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:40:44.857 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/073 2023-11-04T05:40:44.883 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/073 2023-11-04T05:40:44.883 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:40:44.884 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:44.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:40:44.894 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-11-04T05:40:45.269 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:40:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:40:45.280 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:40:45.282 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:40:45.282 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:40:55.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:40:55.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:41:05.396 INFO:teuthology.orchestra.run.smithi003.stdout:generic/074 20s 2023-11-04T05:41:05.406 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/074 2023-11-04T05:41:05.406 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:05.409 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:06.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:06.222 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-11-04T05:41:06.935 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:06.945 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:06.946 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:06.947 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:06.947 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:26.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:41:26.083 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:41:26.158 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T05:41:26.157+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 211657) UID: 0 2023-11-04T05:41:26.159 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T05:41:26.157+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 211657) UID: 0 2023-11-04T05:41:38.834 INFO:teuthology.orchestra.run.smithi003.stdout:generic/075 31s 2023-11-04T05:41:38.842 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/075 2023-11-04T05:41:38.843 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:38.845 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:39.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:39.259 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-11-04T05:41:39.759 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:39.774 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:39.775 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:39.777 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:39.777 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:40.311 INFO:teuthology.orchestra.run.smithi003.stdout:generic/076 [not run] require 172.21.15.3:6789:/scratch to be local device 2023-11-04T05:41:40.316 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/076 2023-11-04T05:41:40.316 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/076 2023-11-04T05:41:40.317 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:40.332 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:40.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:40.369 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-11-04T05:41:40.768 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:40.768 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:40.769 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:40.769 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:40.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:44.222 INFO:teuthology.orchestra.run.smithi003.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T05:41:44.232 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/077 2023-11-04T05:41:44.233 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/077 2023-11-04T05:41:44.233 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:44.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:44.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:44.339 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-11-04T05:41:45.108 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:45.113 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:45.114 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:45.116 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:45.116 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:45.807 INFO:teuthology.orchestra.run.smithi003.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2023-11-04T05:41:45.812 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/078 2023-11-04T05:41:45.812 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/078 2023-11-04T05:41:45.812 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:45.816 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:46.176 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:46.177 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-11-04T05:41:46.616 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:46.620 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:46.621 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:46.622 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:46.622 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:47.315 INFO:teuthology.orchestra.run.smithi003.stdout:generic/079 [not run] file system doesn't support chattr +ia 2023-11-04T05:41:47.332 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/079 2023-11-04T05:41:47.333 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/079 2023-11-04T05:41:47.333 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:47.338 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:47.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:47.404 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-11-04T05:41:47.813 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:47.819 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:47.823 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:47.824 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:47.825 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:50.454 INFO:teuthology.orchestra.run.smithi003.stdout:generic/080 2s 2023-11-04T05:41:50.548 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/080 2023-11-04T05:41:50.548 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:50.551 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:50.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:50.596 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-11-04T05:41:51.163 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:51.176 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:51.178 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:51.180 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:51.754 INFO:teuthology.orchestra.run.smithi003.stdout:generic/081 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:41:51.760 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/081 2023-11-04T05:41:51.760 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/081 2023-11-04T05:41:51.760 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:51.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:51.810 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-11-04T05:41:52.287 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:52.299 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:52.300 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:52.302 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:52.859 INFO:teuthology.orchestra.run.smithi003.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T05:41:52.864 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/082 2023-11-04T05:41:52.864 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/082 2023-11-04T05:41:52.864 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:52.902 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:52.902 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-11-04T05:41:53.362 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:53.369 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:53.369 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:53.371 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:53.983 INFO:teuthology.orchestra.run.smithi003.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T05:41:53.990 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/083 2023-11-04T05:41:53.991 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/083 2023-11-04T05:41:53.991 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:41:53.995 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:41:54.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:41:54.044 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-11-04T05:41:54.499 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:41:54.506 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:41:54.508 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:41:54.509 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:41:54.509 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:00.882 INFO:teuthology.orchestra.run.smithi003.stdout:generic/084 6s 2023-11-04T05:42:00.939 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/084 2023-11-04T05:42:00.939 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:42:00.942 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:00.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:42:00.986 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-11-04T05:42:01.474 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:42:01.482 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:42:01.482 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:42:01.485 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:42:01.485 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:02.145 INFO:teuthology.orchestra.run.smithi003.stdout:generic/085 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:42:02.150 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/085 2023-11-04T05:42:02.151 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/085 2023-11-04T05:42:02.151 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:42:02.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:02.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:42:02.175 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-11-04T05:42:02.654 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:42:02.669 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:42:02.669 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:42:02.674 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:42:02.675 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:03.241 INFO:teuthology.orchestra.run.smithi003.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:42:03.252 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/086 2023-11-04T05:42:03.252 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/086 2023-11-04T05:42:03.253 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:42:03.254 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:03.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:42:03.313 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-11-04T05:42:03.755 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:42:03.763 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:42:03.763 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:42:03.765 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:42:03.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:04.526 INFO:teuthology.orchestra.run.smithi003.stdout:generic/087 0s 2023-11-04T05:42:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/087 2023-11-04T05:42:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:42:04.541 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:04.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:42:04.820 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-11-04T05:42:05.435 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:42:05.451 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:42:05.451 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:42:05.453 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:42:05.453 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:06.092 INFO:teuthology.orchestra.run.smithi003.stdout:generic/088 1s 2023-11-04T05:42:06.126 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/088 2023-11-04T05:42:06.127 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:42:06.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:42:06.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:42:06.500 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-11-04T05:42:07.049 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:42:07.058 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:42:07.059 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:42:07.060 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:42:07.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:44:27.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:44:27.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:44:58.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:44:58.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:45:28.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:45:28.284 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:45:58.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:45:58.392 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:45:58.565 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T05:45:58.564+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 220766) UID: 0 2023-11-04T05:45:58.566 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T05:45:58.564+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 220766) UID: 0 2023-11-04T05:48:50.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:48:50.865 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:49:21.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:49:21.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:49:51.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:49:51.155 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:49:51.190 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T05:49:51.189+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 222880) UID: 0 2023-11-04T05:49:51.190 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T05:49:51.189+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 222880) UID: 0 2023-11-04T05:50:09.769 INFO:teuthology.orchestra.run.smithi003.stdout:generic/089 482s 2023-11-04T05:50:09.774 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/089 2023-11-04T05:50:09.774 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:50:09.776 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:50:12.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:50:12.057 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2023-11-04T05:50:13.011 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:50:13.027 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:50:13.035 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:50:13.036 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:50:13.036 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:50:13.501 INFO:teuthology.orchestra.run.smithi003.stdout:generic/090 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:50:13.505 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/090 2023-11-04T05:50:13.505 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/090 2023-11-04T05:50:13.505 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:50:13.507 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:50:13.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:50:13.542 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2023-11-04T05:50:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:50:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:50:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:50:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:50:13.771 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:04.063 INFO:teuthology.orchestra.run.smithi003.stdout:generic/091 111s 2023-11-04T05:52:04.068 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/091 2023-11-04T05:52:04.068 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:04.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:04.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:04.384 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2023-11-04T05:52:04.673 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:04.677 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:04.678 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:04.679 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:04.679 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:05.014 INFO:teuthology.orchestra.run.smithi003.stdout:generic/092 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:52:05.019 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/092 2023-11-04T05:52:05.019 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/092 2023-11-04T05:52:05.019 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:05.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:05.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:05.047 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2023-11-04T05:52:05.268 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:05.272 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:05.273 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:05.274 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:05.274 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:05.861 INFO:teuthology.orchestra.run.smithi003.stdout:generic/093 0s 2023-11-04T05:52:05.866 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/093 2023-11-04T05:52:05.866 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:05.868 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:06.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:06.204 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2023-11-04T05:52:06.461 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:06.466 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:06.466 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:06.467 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:06.859 INFO:teuthology.orchestra.run.smithi003.stdout:generic/094 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T05:52:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/094 2023-11-04T05:52:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/094 2023-11-04T05:52:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:06.865 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:07.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:07.226 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2023-11-04T05:52:07.475 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:07.479 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:07.479 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:07.480 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:07.480 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:20.732 INFO:teuthology.orchestra.run.smithi003.stdout:generic/095 13s 2023-11-04T05:52:20.737 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/095 2023-11-04T05:52:20.738 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:20.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:20.750 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2023-11-04T05:52:20.972 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:20.976 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:20.977 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:20.978 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:21.414 INFO:teuthology.orchestra.run.smithi003.stdout:generic/096 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-11-04T05:52:21.418 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/096 2023-11-04T05:52:21.419 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/096 2023-11-04T05:52:21.419 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:21.421 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:21.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:21.449 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2023-11-04T05:52:21.672 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:21.676 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:21.677 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:21.677 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:21.678 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:22.540 INFO:teuthology.orchestra.run.smithi003.stdout:generic/097 1s 2023-11-04T05:52:22.545 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/097 2023-11-04T05:52:22.545 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:22.547 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:22.574 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:22.575 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2023-11-04T05:52:22.795 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:22.799 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:22.799 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:22.800 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:22.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:25.172 INFO:teuthology.orchestra.run.smithi003.stdout:generic/098 2s 2023-11-04T05:52:25.177 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/098 2023-11-04T05:52:25.178 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:25.179 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:25.188 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:25.188 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2023-11-04T05:52:25.408 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:25.412 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:25.413 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:25.414 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:25.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:26.390 INFO:teuthology.orchestra.run.smithi003.stdout:generic/099 1s 2023-11-04T05:52:26.395 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/099 2023-11-04T05:52:26.395 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:26.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:26.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:26.566 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2023-11-04T05:52:27.002 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:27.006 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:27.007 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:27.007 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:27.008 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:27.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:52:27.382 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:52:39.556 INFO:teuthology.orchestra.run.smithi003.stdout:generic/100 12s 2023-11-04T05:52:39.565 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/100 2023-11-04T05:52:39.566 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:39.569 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:43.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:43.121 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2023-11-04T05:52:43.443 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:43.448 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:43.448 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:43.449 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:43.798 INFO:teuthology.orchestra.run.smithi003.stdout:generic/101 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:52:43.802 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/101 2023-11-04T05:52:43.803 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/101 2023-11-04T05:52:43.803 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:43.805 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:43.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:43.829 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2023-11-04T05:52:44.038 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:44.042 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:44.042 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:44.043 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:44.043 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:44.370 INFO:teuthology.orchestra.run.smithi003.stdout:generic/102 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T05:52:44.374 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/102 2023-11-04T05:52:44.374 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/102 2023-11-04T05:52:44.375 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:44.376 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:44.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:44.403 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2023-11-04T05:52:44.627 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:44.631 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:44.631 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:44.632 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:44.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:44.977 INFO:teuthology.orchestra.run.smithi003.stdout:generic/103 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T05:52:44.980 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/103 2023-11-04T05:52:44.981 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/103 2023-11-04T05:52:44.981 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:44.983 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:45.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:45.035 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2023-11-04T05:52:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:45.578 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:45.578 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:45.579 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:45.933 INFO:teuthology.orchestra.run.smithi003.stdout:generic/104 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:52:45.937 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/104 2023-11-04T05:52:45.937 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/104 2023-11-04T05:52:45.938 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:45.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:45.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:45.975 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2023-11-04T05:52:46.197 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:46.201 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:46.201 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:46.202 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:46.203 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:47.114 INFO:teuthology.orchestra.run.smithi003.stdout:generic/105 1s 2023-11-04T05:52:47.141 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/105 2023-11-04T05:52:47.142 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:47.145 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:47.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:47.155 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2023-11-04T05:52:47.370 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:47.374 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:47.375 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:47.376 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:47.376 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:47.732 INFO:teuthology.orchestra.run.smithi003.stdout:generic/106 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:52:47.736 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/106 2023-11-04T05:52:47.736 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/106 2023-11-04T05:52:47.736 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:47.738 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:47.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:47.764 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2023-11-04T05:52:47.982 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:47.987 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:47.987 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:47.988 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:47.988 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:48.353 INFO:teuthology.orchestra.run.smithi003.stdout:generic/107 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:52:48.357 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/107 2023-11-04T05:52:48.357 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/107 2023-11-04T05:52:48.357 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:48.359 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:48.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:48.390 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2023-11-04T05:52:48.612 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:48.617 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:48.617 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:48.618 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:49.078 INFO:teuthology.orchestra.run.smithi003.stdout:generic/108 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T05:52:49.082 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/108 2023-11-04T05:52:49.083 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/108 2023-11-04T05:52:49.083 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:49.084 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:49.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:49.118 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2023-11-04T05:52:49.344 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:52:49.348 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:52:49.349 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:52:49.350 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:52:49.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:57.501 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:52:57.504 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:52:59.586 INFO:teuthology.orchestra.run.smithi003.stdout:generic/109 10s 2023-11-04T05:52:59.599 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/109 2023-11-04T05:52:59.600 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:52:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:52:59.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:52:59.611 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2023-11-04T05:53:00.564 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:00.568 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:00.569 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:00.570 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:00.570 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:20.709 INFO:teuthology.orchestra.run.smithi003.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T05:53:20.713 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/110 2023-11-04T05:53:20.714 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/110 2023-11-04T05:53:20.714 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:21.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:21.047 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2023-11-04T05:53:21.648 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:21.652 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:21.652 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:21.653 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:21.654 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:22.015 INFO:teuthology.orchestra.run.smithi003.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T05:53:22.019 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/111 2023-11-04T05:53:22.019 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/111 2023-11-04T05:53:22.020 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:22.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:22.374 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2023-11-04T05:53:22.620 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:22.624 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:22.624 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:22.625 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:22.625 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:28.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:53:28.337 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:53:37.917 INFO:teuthology.orchestra.run.smithi003.stdout:generic/112 15s 2023-11-04T05:53:38.024 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/112 2023-11-04T05:53:38.025 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:38.027 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:38.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:38.364 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2023-11-04T05:53:38.641 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:38.645 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:38.645 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:38.646 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:38.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:53.594 INFO:teuthology.orchestra.run.smithi003.stdout:generic/113 15s 2023-11-04T05:53:53.599 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/113 2023-11-04T05:53:53.599 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:53.601 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:53.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:53.950 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2023-11-04T05:53:54.357 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:54.362 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:54.362 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:54.363 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:54.363 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:54.768 INFO:teuthology.orchestra.run.smithi003.stdout:generic/114 [not run] device block size: 4096 greater than 512 2023-11-04T05:53:54.772 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/114 2023-11-04T05:53:54.773 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/114 2023-11-04T05:53:54.773 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:54.774 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:54.800 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:54.801 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2023-11-04T05:53:55.032 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:55.037 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:55.037 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:55.038 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:55.038 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:55.407 INFO:teuthology.orchestra.run.smithi003.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T05:53:55.412 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/115 2023-11-04T05:53:55.412 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/115 2023-11-04T05:53:55.412 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:55.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:55.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:55.773 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2023-11-04T05:53:56.024 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:56.028 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:56.028 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:56.029 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:56.030 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:56.394 INFO:teuthology.orchestra.run.smithi003.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T05:53:56.398 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/116 2023-11-04T05:53:56.398 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/116 2023-11-04T05:53:56.399 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:53:56.400 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:56.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:53:56.735 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2023-11-04T05:53:57.011 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:53:57.016 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:53:57.016 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:53:57.017 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:53:57.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:53:58.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:53:58.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:54:13.913 INFO:teuthology.orchestra.run.smithi003.stdout:generic/117 16s 2023-11-04T05:54:13.961 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/117 2023-11-04T05:54:13.961 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:13.964 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:13.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:13.990 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2023-11-04T05:54:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:14.221 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:15.089 INFO:teuthology.orchestra.run.smithi003.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T05:54:15.093 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/118 2023-11-04T05:54:15.093 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/118 2023-11-04T05:54:15.094 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:15.095 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:15.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:15.500 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2023-11-04T05:54:15.728 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:15.732 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:15.733 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:15.734 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:15.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:16.105 INFO:teuthology.orchestra.run.smithi003.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T05:54:16.109 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/119 2023-11-04T05:54:16.109 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/119 2023-11-04T05:54:16.109 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:16.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:16.468 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2023-11-04T05:54:16.731 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:16.735 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:16.736 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:16.737 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:17.072 INFO:teuthology.orchestra.run.smithi003.stdout:generic/120 [not run] atime not maintained by ceph 2023-11-04T05:54:17.076 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/120 2023-11-04T05:54:17.076 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/120 2023-11-04T05:54:17.076 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:17.106 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:17.106 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2023-11-04T05:54:17.324 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:17.328 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:17.329 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:17.330 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:17.330 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:17.697 INFO:teuthology.orchestra.run.smithi003.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T05:54:17.701 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/121 2023-11-04T05:54:17.702 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/121 2023-11-04T05:54:17.702 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:17.703 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:18.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:18.039 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2023-11-04T05:54:18.306 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:18.310 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:18.311 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:18.311 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:18.312 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:18.680 INFO:teuthology.orchestra.run.smithi003.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T05:54:18.684 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/122 2023-11-04T05:54:18.684 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/122 2023-11-04T05:54:18.684 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:18.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:18.780 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2023-11-04T05:54:19.292 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:19.296 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:19.297 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:19.298 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:19.808 INFO:teuthology.orchestra.run.smithi003.stdout:generic/123 0s 2023-11-04T05:54:19.813 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/123 2023-11-04T05:54:19.814 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:20.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:20.181 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2023-11-04T05:54:20.419 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:20.423 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:20.423 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:22.711 INFO:teuthology.orchestra.run.smithi003.stdout:generic/124 2s 2023-11-04T05:54:22.717 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/124 2023-11-04T05:54:22.718 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:54:22.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:23.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:54:23.162 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2023-11-04T05:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:54:23.630 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:54:23.631 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:54:23.632 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:54:23.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:54:29.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:54:29.357 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:54:59.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:54:59.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:55:25.138 INFO:teuthology.orchestra.run.smithi003.stdout:generic/125 62s 2023-11-04T05:55:25.143 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/125 2023-11-04T05:55:25.144 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:55:25.146 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:55:25.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:55:25.303 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2023-11-04T05:55:25.739 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:55:25.743 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:55:25.744 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:55:25.744 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:55:25.745 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:55:26.250 INFO:teuthology.orchestra.run.smithi003.stdout:generic/126 1s 2023-11-04T05:55:26.255 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/126 2023-11-04T05:55:26.255 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T05:55:26.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:55:26.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T05:55:26.624 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2023-11-04T05:55:26.887 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T05:55:26.891 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T05:55:26.891 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T05:55:26.892 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T05:55:26.892 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T05:55:29.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:55:29.945 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:56:00.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:56:00.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:56:30.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:56:30.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:57:00.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:57:00.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:57:30.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:57:30.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:58:00.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:58:00.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:58:30.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:58:30.362 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:59:00.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:59:00.424 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:59:30.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T05:59:30.486 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:00:00.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:00:00.540 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:00:30.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:00:30.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:01:00.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:01:00.713 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:01:30.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:01:30.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:02:00.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:02:00.905 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:02:30.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:02:31.002 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:03:01.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:03:01.074 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:03:31.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:03:31.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:04:01.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:04:01.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:04:31.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:04:31.675 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:04:47.108 INFO:teuthology.orchestra.run.smithi003.stdout:generic/127 560s 2023-11-04T06:04:47.112 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/127 2023-11-04T06:04:47.112 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:04:47.114 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:04:47.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:04:47.458 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2023-11-04T06:04:47.710 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:04:47.714 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:04:47.714 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:04:47.715 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:04:47.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:04:48.857 INFO:teuthology.orchestra.run.smithi003.stdout:generic/128 1s 2023-11-04T06:04:48.862 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/128 2023-11-04T06:04:48.862 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:04:48.864 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:04:48.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:04:48.873 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2023-11-04T06:04:49.091 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:04:49.095 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:04:49.096 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:04:49.097 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:04:49.097 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:05:01.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:05:01.755 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:05:31.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:05:31.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:06:01.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:06:01.992 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:06:32.179 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:06:32.191 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:07:02.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:07:02.318 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:07:32.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:07:32.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:08:02.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:08:02.675 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:08:33.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:08:33.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:09:03.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:09:03.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:09:33.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:09:33.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:10:03.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:10:03.568 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:10:03.742 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T06:10:03.735+0000 7f7dba50c700 -1 received signal: Hangup from (PID: 263134) UID: 0 2023-11-04T06:10:03.746 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T06:10:03.743+0000 7f97f5987700 -1 received signal: Hangup from (PID: 263134) UID: 0 2023-11-04T06:10:03.751 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T06:10:03.745+0000 7f58ab082700 -1 received signal: Hangup from (PID: 263134) UID: 0 2023-11-04T06:10:03.758 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T06:10:03.745+0000 7f1f6ddbb700 -1 received signal: Hangup from (PID: 263134) UID: 0 2023-11-04T06:10:03.838 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T06:10:03.835+0000 7eff274c1700 -1 received signal: Hangup from (PID: 116686) UID: 0 2023-11-04T06:10:03.840 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:10:03.835+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 116686) UID: 0 2023-11-04T06:10:03.863 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:10:03.850+0000 7f8a2900f700 -1 Fail to open '/proc/116686/cmdline' error = (2) No such file or directory 2023-11-04T06:10:03.864 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:10:03.850+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 116686) UID: 0 2023-11-04T06:10:03.864 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T06:10:03.835+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 116686) UID: 0 2023-11-04T06:11:02.502 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:11:02.501+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116705) UID: 0 2023-11-04T06:11:02.503 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T06:11:02.501+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116705) UID: 0 2023-11-04T06:11:02.504 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T06:11:02.501+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116705) UID: 0 2023-11-04T06:11:02.504 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:11:02.501+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116705) UID: 0 2023-11-04T06:14:44.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:14:44.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:14:44.861 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T06:14:44.860+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263198) UID: 0 2023-11-04T06:14:44.862 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T06:14:44.860+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263198) UID: 0 2023-11-04T06:14:44.863 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T06:14:44.860+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263198) UID: 0 2023-11-04T06:14:44.863 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T06:14:44.860+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263198) UID: 0 2023-11-04T06:19:39.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:19:39.149 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:19:39.305 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:19:39.303+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 116840) UID: 0 2023-11-04T06:19:39.305 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T06:19:39.303+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 116840) UID: 0 2023-11-04T06:19:39.308 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T06:19:39.303+0000 7eff274c1700 -1 received signal: Hangup from (PID: 116840) UID: 0 2023-11-04T06:19:39.309 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:19:39.305+0000 7f1044bc0700 -1 Fail to open '/proc/116840/cmdline' error = (2) No such file or directory 2023-11-04T06:19:39.309 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:19:39.305+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 116840) UID: 0 2023-11-04T06:19:39.330 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T06:19:39.329+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263267) UID: 0 2023-11-04T06:19:39.331 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T06:19:39.330+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263267) UID: 0 2023-11-04T06:19:39.331 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T06:19:39.330+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263267) UID: 0 2023-11-04T06:19:39.331 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T06:19:39.330+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 263267) UID: 0 2023-11-04T06:25:42.546 INFO:teuthology.orchestra.run.smithi003.stdout:generic/129 1253s 2023-11-04T06:25:42.603 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/129 2023-11-04T06:25:42.603 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:25:42.605 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:25:42.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:25:42.618 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2023-11-04T06:25:43.277 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:25:43.281 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:25:43.281 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:25:43.282 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:25:43.283 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:25:56.139 INFO:teuthology.orchestra.run.smithi003.stdout:generic/130 13s 2023-11-04T06:25:56.160 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/130 2023-11-04T06:25:56.160 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:25:56.162 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:25:56.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:25:56.175 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2023-11-04T06:25:56.400 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:25:56.404 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:25:56.404 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:25:56.405 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:25:56.405 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:25:57.963 INFO:teuthology.orchestra.run.smithi003.stdout:generic/131 1s 2023-11-04T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/131 2023-11-04T06:25:57.977 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:25:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:25:58.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:25:58.092 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2023-11-04T06:25:58.861 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:25:58.866 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:25:58.867 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:25:58.868 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:25:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:26:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:generic/132 14s 2023-11-04T06:26:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/132 2023-11-04T06:26:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:26:13.692 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:26:13.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:26:13.701 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2023-11-04T06:26:13.927 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:26:13.931 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:26:13.932 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:26:13.933 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:26:13.933 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:27:29.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:27:29.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:27:29.591 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T06:27:29.589+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117879) UID: 0 2023-11-04T06:27:29.592 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:27:29.589+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117879) UID: 0 2023-11-04T06:27:29.592 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:27:29.589+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117879) UID: 0 2023-11-04T06:27:29.592 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T06:27:29.589+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117879) UID: 0 2023-11-04T06:30:28.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:30:28.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:30:36.571 INFO:teuthology.orchestra.run.smithi003.stdout:generic/133 262s 2023-11-04T06:30:36.576 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/133 2023-11-04T06:30:36.576 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:36.578 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:36.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:36.985 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2023-11-04T06:30:37.398 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:37.402 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:37.403 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:37.403 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:37.404 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:37.776 INFO:teuthology.orchestra.run.smithi003.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:37.780 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/134 2023-11-04T06:30:37.781 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/134 2023-11-04T06:30:37.781 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:37.783 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:38.105 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:38.105 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2023-11-04T06:30:38.414 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:38.418 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:38.419 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:38.420 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:38.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:39.554 INFO:teuthology.orchestra.run.smithi003.stdout:generic/135 1s 2023-11-04T06:30:39.559 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/135 2023-11-04T06:30:39.559 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:39.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:39.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:39.570 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2023-11-04T06:30:39.795 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:39.798 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:39.799 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:39.800 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:39.800 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:40.175 INFO:teuthology.orchestra.run.smithi003.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T06:30:40.179 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/136 2023-11-04T06:30:40.179 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/136 2023-11-04T06:30:40.179 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:40.181 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:40.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:40.534 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2023-11-04T06:30:40.806 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:40.810 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:40.811 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:40.812 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:40.812 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:41.188 INFO:teuthology.orchestra.run.smithi003.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:41.192 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/137 2023-11-04T06:30:41.192 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/137 2023-11-04T06:30:41.192 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:41.194 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:41.560 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:41.560 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2023-11-04T06:30:41.815 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:41.819 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:41.820 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:41.821 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:41.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:42.171 INFO:teuthology.orchestra.run.smithi003.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:42.175 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/138 2023-11-04T06:30:42.176 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/138 2023-11-04T06:30:42.176 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:42.177 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:42.500 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:42.500 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2023-11-04T06:30:42.809 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:42.813 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:42.813 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:42.814 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:42.814 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:43.179 INFO:teuthology.orchestra.run.smithi003.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:43.183 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/139 2023-11-04T06:30:43.184 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/139 2023-11-04T06:30:43.184 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:43.185 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:43.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:43.454 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2023-11-04T06:30:43.833 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:43.837 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:43.837 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:43.838 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:43.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:44.208 INFO:teuthology.orchestra.run.smithi003.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:44.213 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/140 2023-11-04T06:30:44.213 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/140 2023-11-04T06:30:44.213 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:44.215 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:44.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:44.602 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2023-11-04T06:30:44.842 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:44.846 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:44.847 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:44.848 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:44.848 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:45.428 INFO:teuthology.orchestra.run.smithi003.stdout:generic/141 0s 2023-11-04T06:30:45.433 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/141 2023-11-04T06:30:45.433 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:45.435 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:45.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:45.445 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2023-11-04T06:30:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:45.681 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:46.043 INFO:teuthology.orchestra.run.smithi003.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:46.047 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/142 2023-11-04T06:30:46.047 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/142 2023-11-04T06:30:46.048 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:46.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:46.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:46.394 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2023-11-04T06:30:46.666 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:46.670 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:46.671 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:46.671 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:46.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:47.035 INFO:teuthology.orchestra.run.smithi003.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:47.040 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/143 2023-11-04T06:30:47.040 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/143 2023-11-04T06:30:47.040 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:47.042 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:47.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:47.431 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2023-11-04T06:30:47.672 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:47.676 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:47.676 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:47.677 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:47.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:48.047 INFO:teuthology.orchestra.run.smithi003.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:48.051 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/144 2023-11-04T06:30:48.051 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/144 2023-11-04T06:30:48.051 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:48.053 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:48.422 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:48.423 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2023-11-04T06:30:48.678 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:48.682 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:48.683 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:48.684 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:48.684 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:49.043 INFO:teuthology.orchestra.run.smithi003.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:49.047 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/145 2023-11-04T06:30:49.047 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/145 2023-11-04T06:30:49.048 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:49.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:49.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:49.395 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2023-11-04T06:30:49.670 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:49.674 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:49.674 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:49.675 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:49.675 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:50.049 INFO:teuthology.orchestra.run.smithi003.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:50.053 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/146 2023-11-04T06:30:50.053 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/146 2023-11-04T06:30:50.054 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:50.055 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:50.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:50.364 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2023-11-04T06:30:50.676 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:50.680 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:50.680 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:50.681 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:50.682 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:51.073 INFO:teuthology.orchestra.run.smithi003.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:51.077 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/147 2023-11-04T06:30:51.077 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/147 2023-11-04T06:30:51.077 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:51.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:51.316 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2023-11-04T06:30:51.684 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:51.688 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:51.689 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:51.689 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:52.061 INFO:teuthology.orchestra.run.smithi003.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:52.065 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/148 2023-11-04T06:30:52.066 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/148 2023-11-04T06:30:52.066 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:52.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:52.354 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2023-11-04T06:30:52.682 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:52.686 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:52.687 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:52.688 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:53.058 INFO:teuthology.orchestra.run.smithi003.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:53.062 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/149 2023-11-04T06:30:53.063 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/149 2023-11-04T06:30:53.063 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:53.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:53.371 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2023-11-04T06:30:53.671 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:53.675 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:53.676 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:53.677 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:53.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:54.045 INFO:teuthology.orchestra.run.smithi003.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:54.049 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/150 2023-11-04T06:30:54.049 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/150 2023-11-04T06:30:54.049 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:54.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:54.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:54.351 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2023-11-04T06:30:54.670 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:54.674 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:54.675 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:54.676 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:54.676 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:55.037 INFO:teuthology.orchestra.run.smithi003.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:55.041 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/151 2023-11-04T06:30:55.041 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/151 2023-11-04T06:30:55.041 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:55.043 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:55.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:55.235 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2023-11-04T06:30:55.671 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:55.675 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:55.676 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:55.677 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:55.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:56.053 INFO:teuthology.orchestra.run.smithi003.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:56.057 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/152 2023-11-04T06:30:56.057 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/152 2023-11-04T06:30:56.058 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:56.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:56.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:56.441 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2023-11-04T06:30:56.704 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:56.708 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:56.708 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:56.709 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:56.709 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:57.063 INFO:teuthology.orchestra.run.smithi003.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:57.067 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/153 2023-11-04T06:30:57.068 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/153 2023-11-04T06:30:57.068 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:57.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:57.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:57.182 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2023-11-04T06:30:57.689 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:57.693 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:57.694 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:57.695 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:58.061 INFO:teuthology.orchestra.run.smithi003.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:58.065 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/154 2023-11-04T06:30:58.066 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/154 2023-11-04T06:30:58.066 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:58.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:58.383 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2023-11-04T06:30:58.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:30:58.492 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:30:58.693 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:58.698 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:58.698 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:58.699 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:59.063 INFO:teuthology.orchestra.run.smithi003.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:30:59.068 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/155 2023-11-04T06:30:59.068 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/155 2023-11-04T06:30:59.068 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:30:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:30:59.469 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:30:59.470 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2023-11-04T06:30:59.706 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:30:59.711 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:30:59.711 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:30:59.712 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:30:59.712 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:00.080 INFO:teuthology.orchestra.run.smithi003.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:31:00.084 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/156 2023-11-04T06:31:00.085 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/156 2023-11-04T06:31:00.085 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:00.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:00.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:00.444 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2023-11-04T06:31:00.722 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:00.725 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:00.726 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:00.727 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:31:01.103 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/157 2023-11-04T06:31:01.104 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/157 2023-11-04T06:31:01.104 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:01.105 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:01.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:01.488 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2023-11-04T06:31:01.750 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:01.755 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:01.755 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:01.756 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:01.756 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:02.131 INFO:teuthology.orchestra.run.smithi003.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T06:31:02.135 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/158 2023-11-04T06:31:02.135 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/158 2023-11-04T06:31:02.135 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:02.137 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:02.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:02.312 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2023-11-04T06:31:02.752 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:02.756 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:02.757 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:02.758 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:02.758 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:03.141 INFO:teuthology.orchestra.run.smithi003.stdout:generic/159 [not run] file system doesn't support chattr +i 2023-11-04T06:31:03.145 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/159 2023-11-04T06:31:03.145 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/159 2023-11-04T06:31:03.145 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:03.178 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:03.179 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2023-11-04T06:31:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:03.410 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:03.778 INFO:teuthology.orchestra.run.smithi003.stdout:generic/160 [not run] file system doesn't support chattr +i 2023-11-04T06:31:03.782 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/160 2023-11-04T06:31:03.783 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/160 2023-11-04T06:31:03.783 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:03.784 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:03.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:03.810 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2023-11-04T06:31:04.041 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:04.045 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:04.045 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:04.046 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:04.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:04.884 INFO:teuthology.orchestra.run.smithi003.stdout:generic/161 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:04.894 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/161 2023-11-04T06:31:04.895 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/161 2023-11-04T06:31:04.895 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:04.903 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:04.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:04.938 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2023-11-04T06:31:05.205 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:05.209 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:05.210 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:05.211 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:05.211 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:05.761 INFO:teuthology.orchestra.run.smithi003.stdout:generic/162 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-11-04T06:31:06.051 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/162 2023-11-04T06:31:06.051 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/162 2023-11-04T06:31:06.051 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:06.164 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:06.165 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2023-11-04T06:31:06.401 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:06.405 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:06.406 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:06.407 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:06.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:06.939 INFO:teuthology.orchestra.run.smithi003.stdout:generic/163 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-11-04T06:31:07.231 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/163 2023-11-04T06:31:07.231 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/163 2023-11-04T06:31:07.232 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:07.235 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:07.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:07.330 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2023-11-04T06:31:07.562 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:07.566 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:07.567 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:07.567 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:07.567 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:08.259 INFO:teuthology.orchestra.run.smithi003.stdout:generic/164 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:08.442 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/164 2023-11-04T06:31:08.443 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/164 2023-11-04T06:31:08.443 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:08.449 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:08.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:08.513 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2023-11-04T06:31:08.744 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:08.748 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:08.748 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:08.749 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:generic/165 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/165 2023-11-04T06:31:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/165 2023-11-04T06:31:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:09.628 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:09.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:09.675 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2023-11-04T06:31:09.903 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:09.907 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:09.908 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:09.909 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:09.909 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:10.769 INFO:teuthology.orchestra.run.smithi003.stdout:generic/166 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:10.801 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/166 2023-11-04T06:31:10.801 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/166 2023-11-04T06:31:10.801 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:10.822 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:10.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:10.863 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2023-11-04T06:31:11.084 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:11.089 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:11.089 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:11.090 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:11.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:11.947 INFO:teuthology.orchestra.run.smithi003.stdout:generic/167 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:11.961 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/167 2023-11-04T06:31:11.962 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/167 2023-11-04T06:31:11.962 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:11.968 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:12.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:12.033 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2023-11-04T06:31:12.256 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:12.260 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:12.260 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:12.261 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:12.262 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:13.079 INFO:teuthology.orchestra.run.smithi003.stdout:generic/168 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:13.113 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/168 2023-11-04T06:31:13.113 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/168 2023-11-04T06:31:13.114 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:13.131 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:13.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:13.187 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2023-11-04T06:31:13.418 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:13.422 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:13.423 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:13.424 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:13.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:generic/169 1s 2023-11-04T06:31:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/169 2023-11-04T06:31:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:14.231 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:14.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:14.240 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2023-11-04T06:31:14.462 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:14.467 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:14.467 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:14.468 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:14.468 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:15.301 INFO:teuthology.orchestra.run.smithi003.stdout:generic/170 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:15.360 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/170 2023-11-04T06:31:15.360 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/170 2023-11-04T06:31:15.360 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:15.368 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:15.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:15.429 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2023-11-04T06:31:15.654 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:15.658 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:15.659 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:15.660 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:15.660 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:16.496 INFO:teuthology.orchestra.run.smithi003.stdout:generic/171 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:16.557 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/171 2023-11-04T06:31:16.557 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/171 2023-11-04T06:31:16.558 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:16.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:16.611 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2023-11-04T06:31:16.842 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:16.846 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:16.847 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:16.848 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:17.717 INFO:teuthology.orchestra.run.smithi003.stdout:generic/172 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:17.764 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/172 2023-11-04T06:31:17.764 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/172 2023-11-04T06:31:17.765 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:17.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:17.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:17.814 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2023-11-04T06:31:18.045 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:18.049 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:18.050 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:18.050 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:18.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:18.824 INFO:teuthology.orchestra.run.smithi003.stdout:generic/173 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:18.831 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/173 2023-11-04T06:31:18.831 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/173 2023-11-04T06:31:18.832 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:18.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:18.998 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2023-11-04T06:31:19.222 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:19.226 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:19.227 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:19.228 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:20.136 INFO:teuthology.orchestra.run.smithi003.stdout:generic/174 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:20.153 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/174 2023-11-04T06:31:20.154 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/174 2023-11-04T06:31:20.154 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:20.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:20.217 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2023-11-04T06:31:20.443 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:20.447 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:20.447 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:20.448 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:21.317 INFO:teuthology.orchestra.run.smithi003.stdout:generic/175 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:21.363 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/175 2023-11-04T06:31:21.364 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/175 2023-11-04T06:31:21.364 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:21.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:21.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:21.405 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2023-11-04T06:31:21.624 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:21.628 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:21.629 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:21.630 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:21.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:22.410 INFO:teuthology.orchestra.run.smithi003.stdout:generic/176 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:22.504 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/176 2023-11-04T06:31:22.505 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/176 2023-11-04T06:31:22.505 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:22.533 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:22.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:22.572 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2023-11-04T06:31:22.798 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:22.802 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:22.803 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:22.804 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:22.804 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:23.313 INFO:teuthology.orchestra.run.smithi003.stdout:generic/177 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T06:31:23.317 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/177 2023-11-04T06:31:23.317 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/177 2023-11-04T06:31:23.317 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:23.319 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:23.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:23.637 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2023-11-04T06:31:23.949 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:23.953 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:23.954 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:23.955 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:24.333 INFO:teuthology.orchestra.run.smithi003.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:31:24.337 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/178 2023-11-04T06:31:24.338 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/178 2023-11-04T06:31:24.338 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:24.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:24.651 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2023-11-04T06:31:24.950 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:24.954 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:24.954 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:24.955 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:24.955 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:25.316 INFO:teuthology.orchestra.run.smithi003.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:31:25.320 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/179 2023-11-04T06:31:25.321 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/179 2023-11-04T06:31:25.321 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:25.322 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:25.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:25.706 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2023-11-04T06:31:25.954 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:25.958 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:25.958 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:25.959 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:25.959 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:26.329 INFO:teuthology.orchestra.run.smithi003.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:31:26.333 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/180 2023-11-04T06:31:26.334 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/180 2023-11-04T06:31:26.334 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:26.336 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:26.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:26.713 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2023-11-04T06:31:26.966 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:26.970 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:26.971 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:26.972 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:26.972 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:27.330 INFO:teuthology.orchestra.run.smithi003.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:31:27.334 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/181 2023-11-04T06:31:27.334 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/181 2023-11-04T06:31:27.335 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:27.336 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:27.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:27.683 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2023-11-04T06:31:27.977 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:27.981 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:27.982 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:27.983 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:27.983 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T06:31:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/182 2023-11-04T06:31:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/182 2023-11-04T06:31:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:28.352 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:28.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:31:28.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:31:28.729 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:28.730 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2023-11-04T06:31:29.012 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:29.017 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:29.017 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:29.018 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:29.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:29.673 INFO:teuthology.orchestra.run.smithi003.stdout:generic/183 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:29.903 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/183 2023-11-04T06:31:29.903 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/183 2023-11-04T06:31:29.903 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:29.933 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:29.958 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:29.959 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2023-11-04T06:31:30.183 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:30.187 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:30.187 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:30.188 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:30.189 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:30.556 INFO:teuthology.orchestra.run.smithi003.stdout:generic/184 0s 2023-11-04T06:31:30.561 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/184 2023-11-04T06:31:30.561 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:30.563 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:30.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:30.599 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2023-11-04T06:31:30.821 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:30.825 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:30.825 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:30.826 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:30.826 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:31.709 INFO:teuthology.orchestra.run.smithi003.stdout:generic/185 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:31.719 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/185 2023-11-04T06:31:31.719 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/185 2023-11-04T06:31:31.719 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:31.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:31.768 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2023-11-04T06:31:31.995 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:31.999 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:31.999 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:32.000 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:32.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:32.751 INFO:teuthology.orchestra.run.smithi003.stdout:generic/186 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:32.763 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/186 2023-11-04T06:31:32.764 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/186 2023-11-04T06:31:32.764 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:32.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:32.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:32.962 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2023-11-04T06:31:33.179 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:33.184 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:33.185 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:33.185 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:33.186 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:34.072 INFO:teuthology.orchestra.run.smithi003.stdout:generic/187 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:34.097 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/187 2023-11-04T06:31:34.098 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/187 2023-11-04T06:31:34.098 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:34.102 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:34.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:34.145 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2023-11-04T06:31:34.366 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:34.370 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:34.371 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:34.371 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:34.372 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:35.295 INFO:teuthology.orchestra.run.smithi003.stdout:generic/188 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:35.302 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/188 2023-11-04T06:31:35.302 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/188 2023-11-04T06:31:35.302 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:35.312 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:35.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:35.350 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2023-11-04T06:31:35.575 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:35.579 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:35.580 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:35.581 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:35.581 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:36.527 INFO:teuthology.orchestra.run.smithi003.stdout:generic/189 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:36.545 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/189 2023-11-04T06:31:36.545 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/189 2023-11-04T06:31:36.546 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:36.550 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:36.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:36.575 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2023-11-04T06:31:36.796 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:36.800 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:36.801 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:36.801 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:36.802 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:37.647 INFO:teuthology.orchestra.run.smithi003.stdout:generic/190 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:37.660 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/190 2023-11-04T06:31:37.661 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/190 2023-11-04T06:31:37.661 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:37.664 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:37.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:37.762 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2023-11-04T06:31:37.991 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:37.995 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:37.996 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:37.997 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:37.997 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:38.882 INFO:teuthology.orchestra.run.smithi003.stdout:generic/191 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:38.889 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/191 2023-11-04T06:31:38.890 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/191 2023-11-04T06:31:38.890 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:38.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:38.961 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2023-11-04T06:31:39.183 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:39.187 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:39.187 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:39.188 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:39.188 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:39.531 INFO:teuthology.orchestra.run.smithi003.stdout:generic/192 [not run] atime not maintained by ceph 2023-11-04T06:31:39.535 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/192 2023-11-04T06:31:39.535 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/192 2023-11-04T06:31:39.535 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:39.537 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:39.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:39.564 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2023-11-04T06:31:39.788 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:39.792 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:39.793 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:39.794 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:39.794 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:40.668 INFO:teuthology.orchestra.run.smithi003.stdout:generic/193 1s 2023-11-04T06:31:40.672 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/193 2023-11-04T06:31:40.673 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:40.675 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:40.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:40.952 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2023-11-04T06:31:41.316 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:41.320 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:41.320 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:41.321 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:41.322 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:42.130 INFO:teuthology.orchestra.run.smithi003.stdout:generic/194 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:42.193 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/194 2023-11-04T06:31:42.193 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/194 2023-11-04T06:31:42.193 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:42.197 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:42.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:42.228 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2023-11-04T06:31:42.469 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:42.473 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:42.474 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:42.474 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:42.475 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:43.357 INFO:teuthology.orchestra.run.smithi003.stdout:generic/195 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:43.416 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/195 2023-11-04T06:31:43.417 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/195 2023-11-04T06:31:43.417 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:43.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:43.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:43.449 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2023-11-04T06:31:43.668 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:43.672 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:43.673 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:43.674 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:43.674 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:44.594 INFO:teuthology.orchestra.run.smithi003.stdout:generic/196 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:44.604 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/196 2023-11-04T06:31:44.604 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/196 2023-11-04T06:31:44.604 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:44.608 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:44.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:44.672 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2023-11-04T06:31:44.895 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:44.899 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:44.900 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:44.900 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:44.901 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:generic/197 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/197 2023-11-04T06:31:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/197 2023-11-04T06:31:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:45.782 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:45.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:45.874 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2023-11-04T06:31:46.096 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:46.100 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:46.100 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:46.101 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:46.102 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:46.862 INFO:teuthology.orchestra.run.smithi003.stdout:generic/198 0s 2023-11-04T06:31:46.867 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/198 2023-11-04T06:31:46.867 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:46.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:47.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:47.136 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2023-11-04T06:31:47.489 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:47.494 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:47.494 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:47.495 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:47.495 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:48.402 INFO:teuthology.orchestra.run.smithi003.stdout:generic/199 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:48.441 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/199 2023-11-04T06:31:48.441 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/199 2023-11-04T06:31:48.442 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:48.447 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:48.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:48.521 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2023-11-04T06:31:48.745 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:48.749 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:48.750 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:48.751 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:48.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:49.612 INFO:teuthology.orchestra.run.smithi003.stdout:generic/200 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:49.668 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/200 2023-11-04T06:31:49.668 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/200 2023-11-04T06:31:49.668 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:49.673 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:49.703 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:49.704 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2023-11-04T06:31:49.933 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:49.937 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:49.937 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:49.938 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:49.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:50.897 INFO:teuthology.orchestra.run.smithi003.stdout:generic/201 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:50.913 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/201 2023-11-04T06:31:50.914 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/201 2023-11-04T06:31:50.914 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:50.915 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:50.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:50.940 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2023-11-04T06:31:51.173 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:51.177 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:51.177 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:51.178 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:52.054 INFO:teuthology.orchestra.run.smithi003.stdout:generic/202 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:52.061 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/202 2023-11-04T06:31:52.061 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/202 2023-11-04T06:31:52.061 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:52.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:52.144 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2023-11-04T06:31:52.373 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:52.377 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:52.378 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:52.379 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:53.162 INFO:teuthology.orchestra.run.smithi003.stdout:generic/203 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:53.185 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/203 2023-11-04T06:31:53.185 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/203 2023-11-04T06:31:53.186 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:53.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:53.338 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2023-11-04T06:31:53.568 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:53.572 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:53.573 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:53.574 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:53.574 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:55.467 INFO:teuthology.orchestra.run.smithi003.stdout:generic/204 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:31:55.472 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/204 2023-11-04T06:31:55.472 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/204 2023-11-04T06:31:55.472 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:55.475 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:55.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:55.502 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2023-11-04T06:31:55.740 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:55.744 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:55.744 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:55.745 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:55.746 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:56.678 INFO:teuthology.orchestra.run.smithi003.stdout:generic/205 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:56.685 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/205 2023-11-04T06:31:56.685 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/205 2023-11-04T06:31:56.685 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:56.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:56.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:56.726 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2023-11-04T06:31:56.949 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:56.954 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:56.955 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:56.955 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:56.955 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:57.808 INFO:teuthology.orchestra.run.smithi003.stdout:generic/206 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:31:57.883 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/206 2023-11-04T06:31:57.884 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/206 2023-11-04T06:31:57.884 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:31:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:57.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:31:57.936 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2023-11-04T06:31:58.156 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:31:58.160 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:31:58.161 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:31:58.162 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:31:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:31:58.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:31:58.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:32:04.456 INFO:teuthology.orchestra.run.smithi003.stdout:generic/207 6s 2023-11-04T06:32:04.460 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/207 2023-11-04T06:32:04.460 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:32:04.462 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:32:04.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:32:04.713 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2023-11-04T06:32:05.093 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:32:05.097 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:32:05.098 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:32:05.099 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:32:05.099 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:32:28.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:32:28.957 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:32:59.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:32:59.023 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:33:29.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:33:29.463 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:33:59.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:33:59.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:34:29.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:34:29.699 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:34:59.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:34:59.771 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:35:25.541 INFO:teuthology.orchestra.run.smithi003.stdout:generic/208 200s 2023-11-04T06:35:25.545 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/208 2023-11-04T06:35:25.546 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:35:25.547 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:35:25.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:35:25.899 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2023-11-04T06:35:26.215 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:35:26.219 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:35:26.220 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:35:26.221 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:35:26.221 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:35:29.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:35:29.866 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:36:00.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:36:00.186 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:36:04.254 INFO:teuthology.orchestra.run.smithi003.stdout:generic/209 38s 2023-11-04T06:36:04.259 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/209 2023-11-04T06:36:04.259 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:04.261 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:04.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:04.623 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2023-11-04T06:36:04.888 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:04.892 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:04.892 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:04.893 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:04.893 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:05.333 INFO:teuthology.orchestra.run.smithi003.stdout:generic/210 0s 2023-11-04T06:36:05.338 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/210 2023-11-04T06:36:05.339 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:05.340 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:05.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:05.691 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2023-11-04T06:36:05.961 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:05.966 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:05.966 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:06.482 INFO:teuthology.orchestra.run.smithi003.stdout:generic/211 0s 2023-11-04T06:36:06.486 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/211 2023-11-04T06:36:06.486 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:06.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:06.516 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2023-11-04T06:36:06.736 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:06.740 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:06.741 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:06.742 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:06.742 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:07.175 INFO:teuthology.orchestra.run.smithi003.stdout:generic/212 1s 2023-11-04T06:36:07.180 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/212 2023-11-04T06:36:07.180 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:07.182 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:07.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:07.218 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2023-11-04T06:36:07.440 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:07.444 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:07.445 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:07.446 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:07.446 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:07.793 INFO:teuthology.orchestra.run.smithi003.stdout:generic/213 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:36:07.797 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/213 2023-11-04T06:36:07.798 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/213 2023-11-04T06:36:07.798 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:07.799 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:07.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:07.824 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2023-11-04T06:36:08.045 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:08.049 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:08.050 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:08.051 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:08.416 INFO:teuthology.orchestra.run.smithi003.stdout:generic/214 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:36:08.419 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/214 2023-11-04T06:36:08.420 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/214 2023-11-04T06:36:08.420 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:08.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:08.455 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2023-11-04T06:36:08.680 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:08.684 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:08.685 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:08.686 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:08.686 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:11.111 INFO:teuthology.orchestra.run.smithi003.stdout:generic/215 3s 2023-11-04T06:36:11.117 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/215 2023-11-04T06:36:11.117 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:11.119 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:11.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:11.163 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2023-11-04T06:36:11.397 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:11.401 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:11.402 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:11.403 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:11.403 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:12.294 INFO:teuthology.orchestra.run.smithi003.stdout:generic/216 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:12.307 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/216 2023-11-04T06:36:12.308 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/216 2023-11-04T06:36:12.308 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:12.317 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:12.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:12.356 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2023-11-04T06:36:12.578 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:12.582 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:12.582 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:12.583 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:12.584 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:13.253 INFO:teuthology.orchestra.run.smithi003.stdout:generic/217 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:13.259 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/217 2023-11-04T06:36:13.259 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/217 2023-11-04T06:36:13.259 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:13.262 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:13.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:13.520 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2023-11-04T06:36:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:13.749 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:14.651 INFO:teuthology.orchestra.run.smithi003.stdout:generic/218 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:14.689 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/218 2023-11-04T06:36:14.689 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/218 2023-11-04T06:36:14.690 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:14.694 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:14.719 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:14.720 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2023-11-04T06:36:14.947 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:14.951 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:14.951 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:14.952 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:14.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:15.303 INFO:teuthology.orchestra.run.smithi003.stdout:generic/219 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:15.307 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/219 2023-11-04T06:36:15.308 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/219 2023-11-04T06:36:15.308 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:15.309 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:15.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:15.332 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2023-11-04T06:36:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:15.556 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:15.556 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:15.557 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:15.557 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:16.377 INFO:teuthology.orchestra.run.smithi003.stdout:generic/220 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:16.389 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/220 2023-11-04T06:36:16.389 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/220 2023-11-04T06:36:16.389 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:16.529 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:16.530 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2023-11-04T06:36:16.760 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:16.765 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:16.765 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:16.766 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:18.139 INFO:teuthology.orchestra.run.smithi003.stdout:generic/221 2s 2023-11-04T06:36:18.145 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/221 2023-11-04T06:36:18.145 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:18.147 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:18.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:18.181 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2023-11-04T06:36:18.408 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:18.412 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:18.412 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:18.413 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:18.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:19.174 INFO:teuthology.orchestra.run.smithi003.stdout:generic/222 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:19.286 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/222 2023-11-04T06:36:19.286 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/222 2023-11-04T06:36:19.286 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:19.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:19.382 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2023-11-04T06:36:19.613 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:19.617 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:19.618 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:19.619 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:19.968 INFO:teuthology.orchestra.run.smithi003.stdout:generic/223 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:36:19.973 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/223 2023-11-04T06:36:19.973 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/223 2023-11-04T06:36:19.973 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:20.003 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:20.004 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2023-11-04T06:36:20.224 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:20.228 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:20.229 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:20.230 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:20.556 INFO:teuthology.orchestra.run.smithi003.stdout:generic/224 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:36:20.560 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/224 2023-11-04T06:36:20.560 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/224 2023-11-04T06:36:20.561 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:20.585 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:20.585 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2023-11-04T06:36:20.810 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:20.813 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:20.814 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:20.815 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:21.214 INFO:teuthology.orchestra.run.smithi003.stdout:generic/225 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T06:36:21.218 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/225 2023-11-04T06:36:21.218 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/225 2023-11-04T06:36:21.218 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:21.220 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:21.480 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:21.481 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2023-11-04T06:36:21.845 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:21.849 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:21.849 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:21.850 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:21.850 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:22.212 INFO:teuthology.orchestra.run.smithi003.stdout:generic/226 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:36:22.216 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/226 2023-11-04T06:36:22.217 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/226 2023-11-04T06:36:22.217 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:22.218 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:22.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:22.250 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2023-11-04T06:36:22.469 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:22.473 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:22.474 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:22.474 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:22.475 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:23.207 INFO:teuthology.orchestra.run.smithi003.stdout:generic/227 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:23.232 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/227 2023-11-04T06:36:23.233 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/227 2023-11-04T06:36:23.233 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:23.236 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:23.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:23.432 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2023-11-04T06:36:23.651 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:23.656 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:23.657 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:23.658 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:23.658 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:24.028 INFO:teuthology.orchestra.run.smithi003.stdout:generic/228 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:36:24.032 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/228 2023-11-04T06:36:24.033 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/228 2023-11-04T06:36:24.033 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:24.058 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:24.058 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2023-11-04T06:36:24.282 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:24.287 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:24.287 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:24.288 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:25.209 INFO:teuthology.orchestra.run.smithi003.stdout:generic/229 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:25.226 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/229 2023-11-04T06:36:25.227 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/229 2023-11-04T06:36:25.227 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:25.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:25.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:25.251 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2023-11-04T06:36:25.479 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:25.483 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:25.483 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:25.484 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:25.484 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:25.822 INFO:teuthology.orchestra.run.smithi003.stdout:generic/230 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:25.826 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/230 2023-11-04T06:36:25.826 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/230 2023-11-04T06:36:25.826 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:25.828 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:25.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:25.852 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2023-11-04T06:36:26.074 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:26.079 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:26.079 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:26.080 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:26.080 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:26.405 INFO:teuthology.orchestra.run.smithi003.stdout:generic/231 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:26.409 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/231 2023-11-04T06:36:26.409 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/231 2023-11-04T06:36:26.410 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:26.411 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:26.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:26.433 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2023-11-04T06:36:26.655 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:26.659 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:26.659 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:26.660 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:26.661 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:27.005 INFO:teuthology.orchestra.run.smithi003.stdout:generic/232 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:27.009 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/232 2023-11-04T06:36:27.009 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/232 2023-11-04T06:36:27.009 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:27.011 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:27.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:27.039 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2023-11-04T06:36:27.259 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:27.264 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:27.264 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:27.265 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:27.266 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:27.593 INFO:teuthology.orchestra.run.smithi003.stdout:generic/233 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:27.597 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/233 2023-11-04T06:36:27.597 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/233 2023-11-04T06:36:27.597 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:27.599 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:27.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:27.626 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2023-11-04T06:36:27.850 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:27.854 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:27.854 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:27.855 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:27.855 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:28.187 INFO:teuthology.orchestra.run.smithi003.stdout:generic/234 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:28.191 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/234 2023-11-04T06:36:28.192 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/234 2023-11-04T06:36:28.192 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:28.193 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:28.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:28.227 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2023-11-04T06:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:28.800 INFO:teuthology.orchestra.run.smithi003.stdout:generic/235 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:36:28.804 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/235 2023-11-04T06:36:28.804 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/235 2023-11-04T06:36:28.804 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:28.806 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:28.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:28.830 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2023-11-04T06:36:29.052 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:29.056 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:29.057 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:29.058 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:29.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:30.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:36:30.339 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:36:30.588 INFO:teuthology.orchestra.run.smithi003.stdout:generic/236 1s 2023-11-04T06:36:30.594 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/236 2023-11-04T06:36:30.594 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:30.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:30.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:30.910 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2023-11-04T06:36:31.220 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:31.224 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:31.225 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:31.226 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:31.226 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:31.652 INFO:teuthology.orchestra.run.smithi003.stdout:generic/237 0s 2023-11-04T06:36:31.657 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/237 2023-11-04T06:36:31.657 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:31.659 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:32.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:32.039 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2023-11-04T06:36:32.286 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:32.289 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:32.290 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:32.291 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:32.291 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:33.083 INFO:teuthology.orchestra.run.smithi003.stdout:generic/238 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:36:33.093 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/238 2023-11-04T06:36:33.093 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/238 2023-11-04T06:36:33.093 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:36:33.096 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:36:33.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:36:33.235 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2023-11-04T06:36:33.483 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:36:33.487 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:36:33.488 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:36:33.489 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:36:33.489 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:37:00.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:37:00.485 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:37:30.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:37:30.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:38:00.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:38:00.987 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:38:29.758 INFO:teuthology.orchestra.run.smithi003.stdout:generic/239 116s 2023-11-04T06:38:29.762 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/239 2023-11-04T06:38:29.763 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:38:29.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:38:30.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:38:30.117 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2023-11-04T06:38:30.397 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:38:30.401 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:38:30.401 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:38:30.402 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:38:30.402 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:38:31.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:38:31.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:38:31.293 INFO:teuthology.orchestra.run.smithi003.stdout:generic/240 1s 2023-11-04T06:38:31.371 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/240 2023-11-04T06:38:31.371 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:38:31.373 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:38:31.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:38:31.848 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2023-11-04T06:38:32.092 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:38:32.096 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:38:32.097 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:01.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:39:01.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:39:01.801 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T06:39:01.799+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 321420) UID: 0 2023-11-04T06:39:01.802 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T06:39:01.799+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 321420) UID: 0 2023-11-04T06:39:45.529 INFO:teuthology.orchestra.run.smithi003.stdout:generic/241 73s 2023-11-04T06:39:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/241 2023-11-04T06:39:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:39:45.606 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:46.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:39:46.133 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2023-11-04T06:39:46.528 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:39:46.532 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:39:46.532 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:39:46.533 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:39:46.533 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:47.516 INFO:teuthology.orchestra.run.smithi003.stdout:generic/242 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:39:47.524 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/242 2023-11-04T06:39:47.524 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/242 2023-11-04T06:39:47.524 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:39:47.527 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:47.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:39:47.629 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2023-11-04T06:39:47.867 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:39:47.871 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:39:47.871 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:39:47.872 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:39:47.872 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:48.633 INFO:teuthology.orchestra.run.smithi003.stdout:generic/243 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:39:48.671 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/243 2023-11-04T06:39:48.672 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/243 2023-11-04T06:39:48.672 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:39:48.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:48.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:39:48.846 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2023-11-04T06:39:49.074 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:39:49.077 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:39:49.078 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:39:49.079 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:39:49.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:49.427 INFO:teuthology.orchestra.run.smithi003.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:39:49.431 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/244 2023-11-04T06:39:49.431 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/244 2023-11-04T06:39:49.432 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:39:49.433 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:49.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:39:49.461 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2023-11-04T06:39:49.686 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:39:49.690 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:39:49.691 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:39:49.691 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:39:49.692 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:50.120 INFO:teuthology.orchestra.run.smithi003.stdout:generic/245 1s 2023-11-04T06:39:50.184 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/245 2023-11-04T06:39:50.185 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:39:50.186 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:50.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:39:50.598 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2023-11-04T06:39:50.886 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:39:50.890 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:39:50.891 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:39:50.892 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:39:50.892 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:51.685 INFO:teuthology.orchestra.run.smithi003.stdout:generic/246 0s 2023-11-04T06:39:51.773 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/246 2023-11-04T06:39:51.773 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:39:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:39:52.164 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:39:52.165 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2023-11-04T06:39:52.455 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:39:52.459 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:39:52.459 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:39:52.460 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:39:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:01.698 INFO:teuthology.orchestra.run.smithi003.stdout:generic/247 9s 2023-11-04T06:40:01.708 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/247 2023-11-04T06:40:01.708 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:01.710 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:01.719 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:01.720 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2023-11-04T06:40:01.946 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:01.951 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:01.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:02.318 INFO:teuthology.orchestra.run.smithi003.stdout:generic/248 0s 2023-11-04T06:40:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/248 2023-11-04T06:40:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:02.347 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:02.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:02.394 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2023-11-04T06:40:02.622 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:02.626 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:02.626 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:02.627 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:02.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:03.230 INFO:teuthology.orchestra.run.smithi003.stdout:generic/249 1s 2023-11-04T06:40:03.235 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/249 2023-11-04T06:40:03.235 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:03.237 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:03.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:03.617 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2023-11-04T06:40:03.913 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:03.917 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:03.918 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:03.919 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:03.919 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:04.362 INFO:teuthology.orchestra.run.smithi003.stdout:generic/250 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:40:04.366 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/250 2023-11-04T06:40:04.367 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/250 2023-11-04T06:40:04.367 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:04.369 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:04.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:04.394 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2023-11-04T06:40:04.619 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:04.623 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:04.624 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:04.624 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:04.625 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:05.140 INFO:teuthology.orchestra.run.smithi003.stdout:generic/251 [not run] FITRIM not supported on /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/251 2023-11-04T06:40:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/251 2023-11-04T06:40:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:05.160 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:05.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:05.184 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2023-11-04T06:40:05.408 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:05.412 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:05.412 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:05.413 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:05.413 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:05.776 INFO:teuthology.orchestra.run.smithi003.stdout:generic/252 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:40:05.780 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/252 2023-11-04T06:40:05.780 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/252 2023-11-04T06:40:05.780 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:05.782 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:05.804 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:05.805 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2023-11-04T06:40:06.029 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:06.033 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:06.034 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:06.035 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:06.035 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:06.944 INFO:teuthology.orchestra.run.smithi003.stdout:generic/253 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:40:06.983 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/253 2023-11-04T06:40:06.983 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/253 2023-11-04T06:40:06.983 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:06.987 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:07.010 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:07.011 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2023-11-04T06:40:07.235 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:07.239 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:07.240 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:07.240 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:07.241 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:08.070 INFO:teuthology.orchestra.run.smithi003.stdout:generic/254 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:40:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/254 2023-11-04T06:40:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/254 2023-11-04T06:40:08.170 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:08.178 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:08.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:08.218 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2023-11-04T06:40:08.443 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:08.447 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:08.448 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:08.449 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:08.449 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:08.900 INFO:teuthology.orchestra.run.smithi003.stdout:generic/255 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:40:08.904 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/255 2023-11-04T06:40:08.904 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/255 2023-11-04T06:40:08.904 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:08.906 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:09.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:09.322 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2023-11-04T06:40:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:09.579 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:10.352 INFO:teuthology.orchestra.run.smithi003.stdout:generic/256 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:40:10.355 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/256 2023-11-04T06:40:10.356 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/256 2023-11-04T06:40:10.356 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:10.358 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:10.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:10.398 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2023-11-04T06:40:10.624 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:10.628 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:10.628 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:10.629 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:10.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:11.552 INFO:teuthology.orchestra.run.smithi003.stdout:generic/257 1s 2023-11-04T06:40:11.557 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/257 2023-11-04T06:40:11.557 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:11.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:11.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:11.988 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2023-11-04T06:40:12.302 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:12.306 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:12.306 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:12.307 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:12.308 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:12.650 INFO:teuthology.orchestra.run.smithi003.stdout:generic/258 [not run] ceph does not support negative timestamps 2023-11-04T06:40:12.654 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/258 2023-11-04T06:40:12.655 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/258 2023-11-04T06:40:12.655 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:12.656 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:12.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:12.757 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2023-11-04T06:40:13.081 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:13.086 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:13.086 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:13.087 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:13.886 INFO:teuthology.orchestra.run.smithi003.stdout:generic/259 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:40:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/259 2023-11-04T06:40:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/259 2023-11-04T06:40:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:14.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:14.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:14.057 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2023-11-04T06:40:14.281 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:14.285 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:14.286 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:14.286 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:14.287 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:14.798 INFO:teuthology.orchestra.run.smithi003.stdout:generic/260 [not run] FITRIM not supported on /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:14.815 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/260 2023-11-04T06:40:14.815 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/260 2023-11-04T06:40:14.815 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:14.817 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:14.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:14.841 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2023-11-04T06:40:15.067 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:15.071 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:15.071 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:15.072 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:15.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:15.972 INFO:teuthology.orchestra.run.smithi003.stdout:generic/261 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:40:15.980 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/261 2023-11-04T06:40:15.981 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/261 2023-11-04T06:40:15.981 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:16.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:16.067 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2023-11-04T06:40:16.293 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:16.297 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:16.297 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:16.298 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:17.192 INFO:teuthology.orchestra.run.smithi003.stdout:generic/262 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:40:17.266 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/262 2023-11-04T06:40:17.267 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/262 2023-11-04T06:40:17.267 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:40:17.272 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:40:17.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:40:17.302 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2023-11-04T06:40:17.524 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:40:17.528 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:40:17.529 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:40:17.529 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:40:17.530 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:generic/263 44s 2023-11-04T06:41:01.219 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/263 2023-11-04T06:41:01.219 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:01.221 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:01.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:01.597 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2023-11-04T06:41:02.240 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:02.244 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:02.244 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:02.245 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:02.246 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:03.110 INFO:teuthology.orchestra.run.smithi003.stdout:generic/264 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:03.218 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/264 2023-11-04T06:41:03.219 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/264 2023-11-04T06:41:03.219 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:03.223 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:03.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:03.248 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2023-11-04T06:41:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:03.473 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:03.474 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:03.475 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:03.475 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:04.381 INFO:teuthology.orchestra.run.smithi003.stdout:generic/265 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:04.396 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/265 2023-11-04T06:41:04.397 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/265 2023-11-04T06:41:04.397 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:04.404 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:04.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:04.449 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2023-11-04T06:41:04.675 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:04.679 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:04.680 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:04.681 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:04.681 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:05.630 INFO:teuthology.orchestra.run.smithi003.stdout:generic/266 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:05.636 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/266 2023-11-04T06:41:05.636 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/266 2023-11-04T06:41:05.636 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:05.639 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:05.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:05.664 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2023-11-04T06:41:05.886 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:05.890 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:05.891 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:05.891 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:05.892 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:06.834 INFO:teuthology.orchestra.run.smithi003.stdout:generic/267 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:06.842 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/267 2023-11-04T06:41:06.842 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/267 2023-11-04T06:41:06.842 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:06.853 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:06.922 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:06.923 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2023-11-04T06:41:07.224 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:07.228 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:07.229 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:07.230 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:07.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:08.088 INFO:teuthology.orchestra.run.smithi003.stdout:generic/268 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:08.154 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/268 2023-11-04T06:41:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/268 2023-11-04T06:41:08.155 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:08.175 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:08.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:08.220 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2023-11-04T06:41:08.444 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:08.448 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:08.448 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:08.449 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:08.449 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:08.791 INFO:teuthology.orchestra.run.smithi003.stdout:generic/269 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:41:08.795 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/269 2023-11-04T06:41:08.795 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/269 2023-11-04T06:41:08.795 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:08.797 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:08.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:08.830 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2023-11-04T06:41:09.053 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:09.057 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:09.058 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:09.059 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:09.399 INFO:teuthology.orchestra.run.smithi003.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:41:09.403 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/270 2023-11-04T06:41:09.403 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/270 2023-11-04T06:41:09.403 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:09.405 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:09.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:09.432 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2023-11-04T06:41:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:09.655 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:10.465 INFO:teuthology.orchestra.run.smithi003.stdout:generic/271 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:10.488 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/271 2023-11-04T06:41:10.489 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/271 2023-11-04T06:41:10.489 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:10.677 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:10.678 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2023-11-04T06:41:10.900 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:10.904 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:10.905 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:10.905 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:10.906 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:11.875 INFO:teuthology.orchestra.run.smithi003.stdout:generic/272 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:11.881 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/272 2023-11-04T06:41:11.881 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/272 2023-11-04T06:41:11.881 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:11.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:11.919 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:11.920 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2023-11-04T06:41:12.147 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:12.151 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:12.152 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:12.153 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:12.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:12.491 INFO:teuthology.orchestra.run.smithi003.stdout:generic/273 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:41:12.495 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/273 2023-11-04T06:41:12.495 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/273 2023-11-04T06:41:12.495 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:12.497 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:12.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:12.528 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2023-11-04T06:41:12.747 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:12.751 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:12.751 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:12.752 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:12.752 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:13.114 INFO:teuthology.orchestra.run.smithi003.stdout:generic/274 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T06:41:13.118 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/274 2023-11-04T06:41:13.119 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/274 2023-11-04T06:41:13.119 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:13.121 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:13.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:13.155 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2023-11-04T06:41:13.393 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:13.397 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:13.398 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:13.399 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:13.399 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:generic/275 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:41:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/275 2023-11-04T06:41:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/275 2023-11-04T06:41:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:13.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:13.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:13.764 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2023-11-04T06:41:13.983 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:13.987 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:13.988 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:13.989 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:13.989 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:14.840 INFO:teuthology.orchestra.run.smithi003.stdout:generic/276 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/276 2023-11-04T06:41:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/276 2023-11-04T06:41:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:14.886 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:14.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:14.938 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2023-11-04T06:41:15.178 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:15.182 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:15.183 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:15.184 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:15.184 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:generic/277 [not run] file system doesn't support chattr +A 2023-11-04T06:41:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/277 2023-11-04T06:41:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/277 2023-11-04T06:41:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:15.572 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:15.607 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:15.608 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2023-11-04T06:41:15.833 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:15.838 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:15.838 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:15.839 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:15.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:16.683 INFO:teuthology.orchestra.run.smithi003.stdout:generic/278 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:16.733 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/278 2023-11-04T06:41:16.733 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/278 2023-11-04T06:41:16.733 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:16.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:16.834 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2023-11-04T06:41:17.059 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:17.063 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:17.063 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:17.064 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:18.013 INFO:teuthology.orchestra.run.smithi003.stdout:generic/279 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:18.019 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/279 2023-11-04T06:41:18.019 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/279 2023-11-04T06:41:18.020 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:18.025 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:18.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:18.062 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2023-11-04T06:41:18.287 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:18.291 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:18.292 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:18.293 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:18.293 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:18.675 INFO:teuthology.orchestra.run.smithi003.stdout:generic/280 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:41:18.679 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/280 2023-11-04T06:41:18.679 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/280 2023-11-04T06:41:18.679 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:18.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:18.701 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2023-11-04T06:41:18.928 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:18.932 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:18.932 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:18.933 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:19.881 INFO:teuthology.orchestra.run.smithi003.stdout:generic/281 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:19.890 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/281 2023-11-04T06:41:19.890 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/281 2023-11-04T06:41:19.891 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:19.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:19.944 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2023-11-04T06:41:20.162 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:20.165 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:20.166 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:20.167 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:20.970 INFO:teuthology.orchestra.run.smithi003.stdout:generic/282 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:21.059 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/282 2023-11-04T06:41:21.059 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/282 2023-11-04T06:41:21.060 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:21.094 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:21.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:21.147 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2023-11-04T06:41:21.375 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:21.379 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:21.379 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:21.380 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:21.381 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:22.272 INFO:teuthology.orchestra.run.smithi003.stdout:generic/283 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:22.321 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/283 2023-11-04T06:41:22.321 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/283 2023-11-04T06:41:22.321 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:22.332 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:22.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:22.367 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2023-11-04T06:41:22.586 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:22.590 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:22.591 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:22.592 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:22.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:23.264 INFO:teuthology.orchestra.run.smithi003.stdout:generic/284 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:23.295 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/284 2023-11-04T06:41:23.295 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/284 2023-11-04T06:41:23.295 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:23.319 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:23.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:23.546 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2023-11-04T06:41:23.790 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:23.794 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:23.794 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:23.795 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:generic/285 2s 2023-11-04T06:41:25.237 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/285 2023-11-04T06:41:25.238 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:25.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:25.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:25.764 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2023-11-04T06:41:26.341 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:26.380 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:26.381 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:26.382 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:26.382 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:27.028 INFO:teuthology.orchestra.run.smithi003.stdout:generic/286 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:41:27.033 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/286 2023-11-04T06:41:27.033 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/286 2023-11-04T06:41:27.034 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:27.040 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:27.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:27.080 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2023-11-04T06:41:27.517 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:27.523 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:27.523 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:27.524 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:27.525 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:28.567 INFO:teuthology.orchestra.run.smithi003.stdout:generic/287 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:28.571 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/287 2023-11-04T06:41:28.571 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/287 2023-11-04T06:41:28.571 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:28.674 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:28.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:28.822 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2023-11-04T06:41:29.242 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:29.249 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:29.249 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:29.251 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:29.251 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:30.142 INFO:teuthology.orchestra.run.smithi003.stdout:generic/288 [not run] FITRIM not supported on /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:30.165 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/288 2023-11-04T06:41:30.165 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/288 2023-11-04T06:41:30.166 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:30.167 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:30.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:30.210 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2023-11-04T06:41:30.612 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:30.620 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:30.620 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:30.621 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:30.622 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:31.846 INFO:teuthology.orchestra.run.smithi003.stdout:generic/289 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:31.962 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/289 2023-11-04T06:41:31.962 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/289 2023-11-04T06:41:31.962 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:31.967 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:32.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:32.004 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2023-11-04T06:41:32.314 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:32.318 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:32.319 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:32.320 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:32.320 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:33.205 INFO:teuthology.orchestra.run.smithi003.stdout:generic/290 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:33.246 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/290 2023-11-04T06:41:33.246 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/290 2023-11-04T06:41:33.247 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:33.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:33.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:33.302 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2023-11-04T06:41:33.525 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:33.529 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:33.530 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:33.531 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:33.531 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:34.474 INFO:teuthology.orchestra.run.smithi003.stdout:generic/291 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:34.482 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/291 2023-11-04T06:41:34.482 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/291 2023-11-04T06:41:34.483 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:34.486 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:34.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:34.536 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2023-11-04T06:41:34.764 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:34.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:35.675 INFO:teuthology.orchestra.run.smithi003.stdout:generic/292 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:35.682 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/292 2023-11-04T06:41:35.682 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/292 2023-11-04T06:41:35.682 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:35.685 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:35.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:35.738 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2023-11-04T06:41:35.964 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:35.968 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:35.968 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:35.969 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:36.820 INFO:teuthology.orchestra.run.smithi003.stdout:generic/293 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:36.939 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/293 2023-11-04T06:41:36.939 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/293 2023-11-04T06:41:36.939 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:36.944 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:36.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:36.979 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2023-11-04T06:41:37.199 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:37.203 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:37.203 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:37.204 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:37.204 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:38.341 INFO:teuthology.orchestra.run.smithi003.stdout:generic/294 1s 2023-11-04T06:41:38.349 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/294 2023-11-04T06:41:38.349 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:38.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:38.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:38.362 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2023-11-04T06:41:38.582 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:38.587 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:38.587 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:38.588 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:38.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:generic/295 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:39.558 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/295 2023-11-04T06:41:39.558 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/295 2023-11-04T06:41:39.559 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:39.600 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:39.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:39.653 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2023-11-04T06:41:39.876 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:39.880 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:39.880 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:39.881 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:39.881 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:40.682 INFO:teuthology.orchestra.run.smithi003.stdout:generic/296 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:40.815 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/296 2023-11-04T06:41:40.816 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/296 2023-11-04T06:41:40.816 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:40.827 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:40.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:40.855 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2023-11-04T06:41:41.081 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:41.086 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:41.086 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:41.087 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:41.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:42.001 INFO:teuthology.orchestra.run.smithi003.stdout:generic/297 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:42.050 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/297 2023-11-04T06:41:42.051 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/297 2023-11-04T06:41:42.051 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:42.055 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:42.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:42.087 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2023-11-04T06:41:42.309 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:42.313 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:42.314 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:42.314 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:42.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:43.044 INFO:teuthology.orchestra.run.smithi003.stdout:generic/298 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:43.050 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/298 2023-11-04T06:41:43.051 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/298 2023-11-04T06:41:43.051 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:43.176 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:43.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:43.274 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2023-11-04T06:41:43.499 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:43.503 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:43.503 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:43.504 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:43.504 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:43.903 INFO:teuthology.orchestra.run.smithi003.stdout:generic/299 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:41:43.907 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/299 2023-11-04T06:41:43.907 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/299 2023-11-04T06:41:43.907 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:43.909 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:43.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:43.941 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2023-11-04T06:41:44.163 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:44.167 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:44.168 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:44.169 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:44.169 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:44.569 INFO:teuthology.orchestra.run.smithi003.stdout:generic/300 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:41:44.573 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/300 2023-11-04T06:41:44.573 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/300 2023-11-04T06:41:44.573 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:44.575 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:44.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:44.611 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2023-11-04T06:41:44.837 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:44.841 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:44.842 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:44.846 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:44.846 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:generic/301 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/301 2023-11-04T06:41:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/301 2023-11-04T06:41:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:45.614 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:45.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:45.814 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2023-11-04T06:41:46.038 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:46.042 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:46.043 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:46.044 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:46.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:46.971 INFO:teuthology.orchestra.run.smithi003.stdout:generic/302 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:46.995 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/302 2023-11-04T06:41:46.996 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/302 2023-11-04T06:41:46.996 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:46.999 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:47.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:47.037 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2023-11-04T06:41:47.261 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:47.265 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:47.266 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:47.267 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:47.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:47.634 INFO:teuthology.orchestra.run.smithi003.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:41:47.638 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/303 2023-11-04T06:41:47.638 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/303 2023-11-04T06:41:47.638 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:47.640 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:48.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:48.049 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2023-11-04T06:41:48.296 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:48.300 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:48.301 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:48.302 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:48.302 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:48.674 INFO:teuthology.orchestra.run.smithi003.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T06:41:48.678 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/304 2023-11-04T06:41:48.678 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/304 2023-11-04T06:41:48.678 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:48.680 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:49.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:49.100 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2023-11-04T06:41:49.352 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:49.358 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:49.358 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:49.358 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:49.359 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:50.305 INFO:teuthology.orchestra.run.smithi003.stdout:generic/305 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:41:50.322 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/305 2023-11-04T06:41:50.323 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/305 2023-11-04T06:41:50.323 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:50.326 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:50.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:50.356 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2023-11-04T06:41:50.575 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:50.579 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:50.580 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:50.581 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:50.581 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:51.765 INFO:teuthology.orchestra.run.smithi003.stdout:generic/306 1s 2023-11-04T06:41:51.778 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/306 2023-11-04T06:41:51.778 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:52.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:52.160 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2023-11-04T06:41:52.424 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:52.428 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:52.429 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:52.429 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:54.252 INFO:teuthology.orchestra.run.smithi003.stdout:generic/307 2s 2023-11-04T06:41:54.257 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/307 2023-11-04T06:41:54.257 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:54.259 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:54.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:54.268 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2023-11-04T06:41:54.489 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:54.492 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:54.493 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:54.494 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:54.494 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:54.873 INFO:teuthology.orchestra.run.smithi003.stdout:generic/308 0s 2023-11-04T06:41:54.878 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/308 2023-11-04T06:41:54.878 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:54.880 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:54.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:54.917 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2023-11-04T06:41:55.138 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:55.143 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:55.143 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:55.144 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:55.145 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:56.568 INFO:teuthology.orchestra.run.smithi003.stdout:generic/309 1s 2023-11-04T06:41:56.573 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/309 2023-11-04T06:41:56.573 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:41:56.575 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:41:56.981 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:41:56.981 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2023-11-04T06:41:57.237 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:41:57.241 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:41:57.242 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:41:57.243 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:41:57.243 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:42:34.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:42:34.052 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:42:34.382 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T06:42:34.381+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119398) UID: 0 2023-11-04T06:42:34.383 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:42:34.381+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119398) UID: 0 2023-11-04T06:42:34.384 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T06:42:34.381+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119398) UID: 0 2023-11-04T06:42:34.384 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:42:34.381+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119398) UID: 0 2023-11-04T06:43:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:generic/310 76s 2023-11-04T06:43:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/310 2023-11-04T06:43:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:13.755 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:14.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:14.145 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2023-11-04T06:43:14.567 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:14.571 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:14.572 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:14.573 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:14.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:14.957 INFO:teuthology.orchestra.run.smithi003.stdout:generic/311 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:43:14.961 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/311 2023-11-04T06:43:14.961 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/311 2023-11-04T06:43:14.962 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:14.963 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:15.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:15.003 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2023-11-04T06:43:15.223 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:15.227 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:15.227 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:15.228 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:15.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:generic/312 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:43:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/312 2023-11-04T06:43:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/312 2023-11-04T06:43:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:15.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:15.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:15.612 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2023-11-04T06:43:15.840 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:15.844 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:15.844 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:15.845 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:20.227 INFO:teuthology.orchestra.run.smithi003.stdout:generic/313 4s 2023-11-04T06:43:20.233 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/313 2023-11-04T06:43:20.233 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:20.235 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:20.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:20.270 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2023-11-04T06:43:20.504 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:20.508 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:20.509 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:20.510 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:20.510 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:21.054 INFO:teuthology.orchestra.run.smithi003.stdout:generic/314 1s 2023-11-04T06:43:21.058 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/314 2023-11-04T06:43:21.059 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:21.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:21.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:21.096 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2023-11-04T06:43:21.325 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:21.329 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:21.330 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:21.331 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:21.331 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:21.683 INFO:teuthology.orchestra.run.smithi003.stdout:generic/315 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T06:43:21.687 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/315 2023-11-04T06:43:21.687 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/315 2023-11-04T06:43:21.687 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:21.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:21.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:21.724 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2023-11-04T06:43:21.948 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:21.952 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:21.953 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:21.953 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:21.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:22.422 INFO:teuthology.orchestra.run.smithi003.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T06:43:22.426 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/316 2023-11-04T06:43:22.426 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/316 2023-11-04T06:43:22.427 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:22.428 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:22.797 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:22.797 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2023-11-04T06:43:23.039 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:23.045 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:23.045 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:23.046 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:23.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:27.116 INFO:teuthology.orchestra.run.smithi003.stdout:generic/317 4s 2023-11-04T06:43:27.121 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/317 2023-11-04T06:43:27.121 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:27.123 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:27.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:27.133 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2023-11-04T06:43:27.355 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:27.359 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:27.360 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:27.361 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:27.361 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:28.185 INFO:teuthology.orchestra.run.smithi003.stdout:generic/318 1s 2023-11-04T06:43:28.190 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/318 2023-11-04T06:43:28.191 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:28.192 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:28.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:28.202 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2023-11-04T06:43:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:28.430 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:29.034 INFO:teuthology.orchestra.run.smithi003.stdout:generic/319 1s 2023-11-04T06:43:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/319 2023-11-04T06:43:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:29.042 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:29.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:29.051 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2023-11-04T06:43:29.273 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:29.275 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:29.276 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:29.277 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:29.277 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:29.626 INFO:teuthology.orchestra.run.smithi003.stdout:generic/320 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:43:29.630 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/320 2023-11-04T06:43:29.630 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/320 2023-11-04T06:43:29.630 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:29.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:29.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:29.659 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2023-11-04T06:43:29.880 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:29.886 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:29.886 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:29.886 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:30.242 INFO:teuthology.orchestra.run.smithi003.stdout:generic/321 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:43:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/321 2023-11-04T06:43:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/321 2023-11-04T06:43:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:30.248 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:30.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:30.274 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2023-11-04T06:43:30.494 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:30.498 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:30.501 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:30.502 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:30.502 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:30.850 INFO:teuthology.orchestra.run.smithi003.stdout:generic/322 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:43:30.854 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/322 2023-11-04T06:43:30.854 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/322 2023-11-04T06:43:30.854 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:43:30.856 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:43:30.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:43:30.883 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2023-11-04T06:43:31.105 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:43:31.110 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:43:31.110 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:43:31.111 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:43:31.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:01.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:45:01.159 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:45:01.207 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T06:45:01.207+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 367891) UID: 0 2023-11-04T06:45:01.208 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T06:45:01.207+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 367891) UID: 0 2023-11-04T06:45:01.209 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T06:45:01.207+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 367891) UID: 0 2023-11-04T06:45:01.210 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T06:45:01.207+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 367891) UID: 0 2023-11-04T06:45:32.092 INFO:teuthology.orchestra.run.smithi003.stdout:generic/323 121s 2023-11-04T06:45:32.096 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/323 2023-11-04T06:45:32.096 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:32.098 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:32.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:32.526 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2023-11-04T06:45:32.754 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:32.758 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:32.758 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:32.759 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:32.760 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:33.090 INFO:teuthology.orchestra.run.smithi003.stdout:generic/324 [not run] defragmentation not supported for fstype "ceph" 2023-11-04T06:45:33.094 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/324 2023-11-04T06:45:33.095 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/324 2023-11-04T06:45:33.095 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:33.097 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:33.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:33.125 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2023-11-04T06:45:33.356 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:33.361 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:33.361 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:33.362 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:33.362 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:generic/325 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:45:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/325 2023-11-04T06:45:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/325 2023-11-04T06:45:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:34.342 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:34.368 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:34.369 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2023-11-04T06:45:34.599 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:34.603 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:34.603 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:34.604 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:34.605 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:35.611 INFO:teuthology.orchestra.run.smithi003.stdout:generic/326 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:35.631 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/326 2023-11-04T06:45:35.631 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/326 2023-11-04T06:45:35.631 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:35.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:35.690 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:35.691 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2023-11-04T06:45:35.914 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:35.919 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:35.919 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:35.920 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:35.920 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:36.726 INFO:teuthology.orchestra.run.smithi003.stdout:generic/327 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:36.859 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/327 2023-11-04T06:45:36.859 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/327 2023-11-04T06:45:36.859 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:36.883 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:36.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:36.921 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2023-11-04T06:45:37.144 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:37.149 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:37.149 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:37.151 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:37.151 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:38.122 INFO:teuthology.orchestra.run.smithi003.stdout:generic/328 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:38.128 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/328 2023-11-04T06:45:38.128 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/328 2023-11-04T06:45:38.128 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:38.130 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:38.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:38.155 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2023-11-04T06:45:38.375 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:38.379 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:38.380 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:38.381 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:38.381 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:39.372 INFO:teuthology.orchestra.run.smithi003.stdout:generic/329 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:39.382 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/329 2023-11-04T06:45:39.382 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/329 2023-11-04T06:45:39.383 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:39.386 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:39.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:39.418 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2023-11-04T06:45:39.641 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:39.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:40.437 INFO:teuthology.orchestra.run.smithi003.stdout:generic/330 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:40.444 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/330 2023-11-04T06:45:40.445 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/330 2023-11-04T06:45:40.445 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:40.446 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:40.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:40.579 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2023-11-04T06:45:40.870 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:40.874 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:40.875 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:40.875 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:40.876 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:41.858 INFO:teuthology.orchestra.run.smithi003.stdout:generic/331 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:41.969 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/331 2023-11-04T06:45:41.969 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/331 2023-11-04T06:45:41.970 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:41.971 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:42.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:42.013 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2023-11-04T06:45:42.231 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:42.236 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:42.236 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:42.237 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:42.237 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:43.141 INFO:teuthology.orchestra.run.smithi003.stdout:generic/332 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:43.210 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/332 2023-11-04T06:45:43.211 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/332 2023-11-04T06:45:43.211 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:43.217 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:43.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:43.253 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2023-11-04T06:45:43.490 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:43.494 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:43.494 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:43.495 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:43.496 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:44.446 INFO:teuthology.orchestra.run.smithi003.stdout:generic/333 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:44.460 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/333 2023-11-04T06:45:44.460 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/333 2023-11-04T06:45:44.461 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:44.462 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:44.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:44.489 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2023-11-04T06:45:44.709 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:44.713 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:44.714 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:44.714 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:44.715 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:generic/334 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:45:45.639 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/334 2023-11-04T06:45:45.639 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/334 2023-11-04T06:45:45.640 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:45.645 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:45.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:45.694 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2023-11-04T06:45:45.920 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:45.924 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:45.925 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:45.926 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:45.926 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:46.288 INFO:teuthology.orchestra.run.smithi003.stdout:generic/335 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:45:46.291 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/335 2023-11-04T06:45:46.292 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/335 2023-11-04T06:45:46.292 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:46.294 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:46.320 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:46.320 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2023-11-04T06:45:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:46.547 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:46.548 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:46.549 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:46.549 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:47.078 INFO:teuthology.orchestra.run.smithi003.stdout:generic/336 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:45:47.082 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/336 2023-11-04T06:45:47.083 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/336 2023-11-04T06:45:47.083 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:47.085 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:47.123 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:47.123 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2023-11-04T06:45:47.343 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:47.347 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:47.348 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:47.348 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:47.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:48.466 INFO:teuthology.orchestra.run.smithi003.stdout:generic/337 1s 2023-11-04T06:45:48.543 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/337 2023-11-04T06:45:48.543 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:48.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:48.554 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:48.555 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2023-11-04T06:45:48.775 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:48.781 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:49.147 INFO:teuthology.orchestra.run.smithi003.stdout:generic/338 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:45:49.151 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/338 2023-11-04T06:45:49.152 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/338 2023-11-04T06:45:49.152 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:45:49.154 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:45:49.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:45:49.180 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2023-11-04T06:45:49.400 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:45:49.404 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:45:49.405 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:45:49.406 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:45:49.406 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:46.144 INFO:teuthology.orchestra.run.smithi003.stdout:generic/339 57s 2023-11-04T06:46:46.158 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/339 2023-11-04T06:46:46.158 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:46:46.160 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:46.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:46:46.169 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2023-11-04T06:46:46.419 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:46:46.423 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:46:46.424 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:46:46.424 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:46:46.425 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:50.092 INFO:teuthology.orchestra.run.smithi003.stdout:generic/340 4s 2023-11-04T06:46:50.223 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/340 2023-11-04T06:46:50.224 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:46:50.225 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:50.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:46:50.235 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2023-11-04T06:46:50.634 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:46:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:46:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:46:50.647 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:46:50.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:51.286 INFO:teuthology.orchestra.run.smithi003.stdout:generic/341 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:46:51.291 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/341 2023-11-04T06:46:51.291 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/341 2023-11-04T06:46:51.292 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:46:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:51.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:46:51.352 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2023-11-04T06:46:51.583 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:46:51.588 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:46:51.588 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:46:51.589 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:46:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:51.947 INFO:teuthology.orchestra.run.smithi003.stdout:generic/342 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:46:51.951 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/342 2023-11-04T06:46:51.951 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/342 2023-11-04T06:46:51.952 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:46:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:51.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:46:51.980 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2023-11-04T06:46:52.208 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:46:52.212 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:46:52.213 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:46:52.213 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:46:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:52.653 INFO:teuthology.orchestra.run.smithi003.stdout:generic/343 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:46:52.657 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/343 2023-11-04T06:46:52.658 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/343 2023-11-04T06:46:52.658 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:46:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:52.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:46:52.694 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2023-11-04T06:46:52.918 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:46:52.922 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:46:52.923 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:46:52.923 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:46:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:58.456 INFO:teuthology.orchestra.run.smithi003.stdout:generic/344 5s 2023-11-04T06:46:58.546 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/344 2023-11-04T06:46:58.546 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:46:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:46:58.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:46:58.557 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2023-11-04T06:46:58.924 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:46:58.931 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:46:58.931 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:46:58.933 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:46:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:04.991 INFO:teuthology.orchestra.run.smithi003.stdout:generic/345 5s 2023-11-04T06:47:05.054 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/345 2023-11-04T06:47:05.054 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:05.057 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:05.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:05.089 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2023-11-04T06:47:06.046 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:06.056 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:06.056 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:06.064 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:09.404 INFO:teuthology.orchestra.run.smithi003.stdout:generic/346 3s 2023-11-04T06:47:09.481 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/346 2023-11-04T06:47:09.481 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:09.488 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:09.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:09.500 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2023-11-04T06:47:09.828 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:09.833 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:09.834 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:09.835 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:09.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:10.502 INFO:teuthology.orchestra.run.smithi003.stdout:generic/347 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:47:10.506 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/347 2023-11-04T06:47:10.506 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/347 2023-11-04T06:47:10.507 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:10.508 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:10.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:10.533 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2023-11-04T06:47:10.757 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:10.761 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:10.762 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:10.763 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:10.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:11.200 INFO:teuthology.orchestra.run.smithi003.stdout:generic/348 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:47:11.204 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/348 2023-11-04T06:47:11.205 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/348 2023-11-04T06:47:11.205 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:11.207 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:11.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:11.241 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2023-11-04T06:47:11.464 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:11.468 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:11.468 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:11.469 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:11.978 INFO:teuthology.orchestra.run.smithi003.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-11-04T06:47:11.982 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/349 2023-11-04T06:47:11.982 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/349 2023-11-04T06:47:11.982 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:11.984 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:12.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:12.015 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2023-11-04T06:47:12.242 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:12.247 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:12.247 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:12.248 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:12.248 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:13.201 INFO:teuthology.orchestra.run.smithi003.stdout:generic/350 1s 2023-11-04T06:47:13.205 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/350 2023-11-04T06:47:13.206 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:13.208 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:13.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:13.217 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2023-11-04T06:47:13.443 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:13.447 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:13.448 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:13.448 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:13.448 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:13.840 INFO:teuthology.orchestra.run.smithi003.stdout:generic/351 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:47:13.844 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/351 2023-11-04T06:47:13.844 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/351 2023-11-04T06:47:13.844 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:13.846 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:13.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:13.875 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2023-11-04T06:47:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:14.105 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:14.892 INFO:teuthology.orchestra.run.smithi003.stdout:generic/352 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:14.898 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/352 2023-11-04T06:47:14.899 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/352 2023-11-04T06:47:14.899 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:14.902 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:15.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:15.090 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2023-11-04T06:47:15.313 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:15.317 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:15.318 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:15.318 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:16.158 INFO:teuthology.orchestra.run.smithi003.stdout:generic/353 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:16.285 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/353 2023-11-04T06:47:16.286 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/353 2023-11-04T06:47:16.286 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:16.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:16.335 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2023-11-04T06:47:16.570 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:16.574 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:16.574 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:16.575 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:21.817 INFO:teuthology.orchestra.run.smithi003.stdout:generic/354 5s 2023-11-04T06:47:21.886 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/354 2023-11-04T06:47:21.886 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:21.891 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:21.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:21.908 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2023-11-04T06:47:22.383 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:22.403 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:22.408 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:22.409 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:22.410 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:23.559 INFO:teuthology.orchestra.run.smithi003.stdout:generic/355 1s 2023-11-04T06:47:23.666 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/355 2023-11-04T06:47:23.666 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:23.668 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:24.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:24.038 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2023-11-04T06:47:24.329 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:24.333 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:24.334 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:24.335 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:25.070 INFO:teuthology.orchestra.run.smithi003.stdout:generic/356 [not run] swapfiles are not supported 2023-11-04T06:47:25.074 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/356 2023-11-04T06:47:25.074 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/356 2023-11-04T06:47:25.074 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:25.076 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:25.108 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:25.108 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2023-11-04T06:47:25.335 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:25.339 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:25.340 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:25.341 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:25.341 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:25.942 INFO:teuthology.orchestra.run.smithi003.stdout:generic/357 [not run] swapfiles are not supported 2023-11-04T06:47:25.946 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/357 2023-11-04T06:47:25.946 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/357 2023-11-04T06:47:25.950 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:25.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:25.985 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2023-11-04T06:47:26.215 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:26.219 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:26.220 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:26.221 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:26.221 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:27.052 INFO:teuthology.orchestra.run.smithi003.stdout:generic/358 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:27.168 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/358 2023-11-04T06:47:27.168 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/358 2023-11-04T06:47:27.168 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:27.202 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:27.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:27.241 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2023-11-04T06:47:27.471 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:27.475 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:27.475 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:27.476 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:27.477 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:28.274 INFO:teuthology.orchestra.run.smithi003.stdout:generic/359 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/359 2023-11-04T06:47:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/359 2023-11-04T06:47:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:28.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:28.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:28.487 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2023-11-04T06:47:28.735 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:28.739 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:28.740 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:28.741 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:28.741 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:29.293 INFO:teuthology.orchestra.run.smithi003.stdout:generic/360 1s 2023-11-04T06:47:29.314 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/360 2023-11-04T06:47:29.314 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:29.316 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:29.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:29.345 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2023-11-04T06:47:29.574 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:29.578 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:29.579 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:29.580 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:29.580 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:29.941 INFO:teuthology.orchestra.run.smithi003.stdout:generic/361 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:47:29.945 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/361 2023-11-04T06:47:29.946 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/361 2023-11-04T06:47:29.946 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:29.947 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:29.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:29.978 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2023-11-04T06:47:30.210 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:30.215 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:30.215 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:30.216 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:30.216 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:30.559 INFO:teuthology.orchestra.run.smithi003.stdout:generic/362 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:30.563 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/362 2023-11-04T06:47:30.563 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/362 2023-11-04T06:47:30.564 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:30.565 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:30.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:30.596 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2023-11-04T06:47:30.823 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:30.827 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:30.827 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:30.828 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:30.828 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:31.176 INFO:teuthology.orchestra.run.smithi003.stdout:generic/363 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:31.178 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/363 2023-11-04T06:47:31.179 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/363 2023-11-04T06:47:31.179 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:31.180 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:31.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:31.210 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2023-11-04T06:47:31.466 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:31.472 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:31.473 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:31.474 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:31.475 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:31.810 INFO:teuthology.orchestra.run.smithi003.stdout:generic/364 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:31.814 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/364 2023-11-04T06:47:31.814 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/364 2023-11-04T06:47:31.815 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:31.816 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:31.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:31.846 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2023-11-04T06:47:32.078 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:32.082 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:32.082 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:32.083 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:32.083 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:32.425 INFO:teuthology.orchestra.run.smithi003.stdout:generic/365 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:32.428 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/365 2023-11-04T06:47:32.428 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/365 2023-11-04T06:47:32.429 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:32.430 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:32.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:32.455 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2023-11-04T06:47:32.680 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:32.686 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:32.686 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:33.035 INFO:teuthology.orchestra.run.smithi003.stdout:generic/366 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:33.038 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/366 2023-11-04T06:47:33.039 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/366 2023-11-04T06:47:33.039 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:33.041 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:33.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:33.067 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2023-11-04T06:47:33.290 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:33.294 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:33.295 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:33.296 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:33.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:33.632 INFO:teuthology.orchestra.run.smithi003.stdout:generic/367 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:33.636 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/367 2023-11-04T06:47:33.637 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/367 2023-11-04T06:47:33.637 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:33.639 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:33.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:33.666 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2023-11-04T06:47:33.889 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:33.893 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:33.894 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:33.895 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:33.895 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:34.242 INFO:teuthology.orchestra.run.smithi003.stdout:generic/368 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:34.246 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/368 2023-11-04T06:47:34.247 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/368 2023-11-04T06:47:34.247 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:34.248 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:34.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:34.274 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2023-11-04T06:47:34.500 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:34.504 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:34.504 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:34.505 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:34.505 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:34.870 INFO:teuthology.orchestra.run.smithi003.stdout:generic/369 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:34.874 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/369 2023-11-04T06:47:34.875 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/369 2023-11-04T06:47:34.875 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:34.877 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:34.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:34.906 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2023-11-04T06:47:35.130 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:35.134 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:35.135 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:35.135 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:35.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:35.473 INFO:teuthology.orchestra.run.smithi003.stdout:generic/370 [not run] this test requires richacl support on $SCRATCH_DEV 2023-11-04T06:47:35.477 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/370 2023-11-04T06:47:35.477 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/370 2023-11-04T06:47:35.477 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:35.479 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:35.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:35.507 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2023-11-04T06:47:35.738 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:35.742 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:35.743 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:35.744 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:35.744 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:generic/371 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:47:36.113 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/371 2023-11-04T06:47:36.114 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/371 2023-11-04T06:47:36.114 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:36.115 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:36.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:36.151 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2023-11-04T06:47:36.382 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:36.390 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:36.391 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:36.391 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:36.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:37.155 INFO:teuthology.orchestra.run.smithi003.stdout:generic/372 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:37.304 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/372 2023-11-04T06:47:37.304 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/372 2023-11-04T06:47:37.304 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:37.310 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:37.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:37.367 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2023-11-04T06:47:37.600 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:37.604 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:37.605 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:37.605 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:37.606 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:generic/373 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/373 2023-11-04T06:47:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/373 2023-11-04T06:47:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:38.574 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:38.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:38.607 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2023-11-04T06:47:38.850 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:38.851 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:38.852 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:38.853 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:38.853 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:39.419 INFO:teuthology.orchestra.run.smithi003.stdout:generic/374 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-11-04T06:47:39.596 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/374 2023-11-04T06:47:39.596 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/374 2023-11-04T06:47:39.597 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:39.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:39.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:39.860 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2023-11-04T06:47:40.096 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:40.101 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:40.101 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:40.102 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:40.102 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:40.581 INFO:teuthology.orchestra.run.smithi003.stdout:generic/375 0s 2023-11-04T06:47:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/375 2023-11-04T06:47:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:40.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:40.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:40.681 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2023-11-04T06:47:41.235 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:41.239 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:41.239 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:41.240 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:41.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:41.623 INFO:teuthology.orchestra.run.smithi003.stdout:generic/376 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:47:41.628 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/376 2023-11-04T06:47:41.628 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/376 2023-11-04T06:47:41.628 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:41.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:41.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:41.654 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2023-11-04T06:47:41.877 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:41.882 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:41.882 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:41.883 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:41.883 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:42.537 INFO:teuthology.orchestra.run.smithi003.stdout:generic/377 0s 2023-11-04T06:47:42.542 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/377 2023-11-04T06:47:42.542 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:42.544 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:42.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:42.553 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2023-11-04T06:47:42.778 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:42.783 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:43.280 INFO:teuthology.orchestra.run.smithi003.stdout:generic/378 0s 2023-11-04T06:47:43.285 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/378 2023-11-04T06:47:43.285 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:43.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:43.489 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:43.490 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2023-11-04T06:47:43.972 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:43.976 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:43.977 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:43.977 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:43.978 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:44.324 INFO:teuthology.orchestra.run.smithi003.stdout:generic/379 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:44.328 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/379 2023-11-04T06:47:44.329 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/379 2023-11-04T06:47:44.329 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:44.331 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:44.359 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:44.359 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2023-11-04T06:47:44.588 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:44.592 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:44.593 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:44.594 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:44.594 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:44.931 INFO:teuthology.orchestra.run.smithi003.stdout:generic/380 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/380 2023-11-04T06:47:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/380 2023-11-04T06:47:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:44.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:44.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:44.968 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2023-11-04T06:47:45.204 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:45.208 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:45.208 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:45.209 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:45.210 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:generic/381 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/381 2023-11-04T06:47:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/381 2023-11-04T06:47:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:45.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:45.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:45.580 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2023-11-04T06:47:45.810 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:45.815 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:45.815 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:45.816 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:46.161 INFO:teuthology.orchestra.run.smithi003.stdout:generic/382 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:46.165 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/382 2023-11-04T06:47:46.165 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/382 2023-11-04T06:47:46.166 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:46.167 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:46.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:46.192 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2023-11-04T06:47:46.423 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:46.427 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:46.427 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:46.428 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:46.428 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:46.775 INFO:teuthology.orchestra.run.smithi003.stdout:generic/383 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:46.779 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/383 2023-11-04T06:47:46.779 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/383 2023-11-04T06:47:46.779 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:46.781 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:46.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:46.809 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2023-11-04T06:47:47.032 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:47.037 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:47.037 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:47.038 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:47.038 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:47.423 INFO:teuthology.orchestra.run.smithi003.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:47.427 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/384 2023-11-04T06:47:47.427 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/384 2023-11-04T06:47:47.427 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:47.429 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:47.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:47.454 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2023-11-04T06:47:47.692 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:47.696 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:47.696 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:47.697 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:47.698 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:48.045 INFO:teuthology.orchestra.run.smithi003.stdout:generic/385 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:48.049 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/385 2023-11-04T06:47:48.049 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/385 2023-11-04T06:47:48.049 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:48.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:48.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:48.080 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2023-11-04T06:47:48.310 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:48.314 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:48.314 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:48.315 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:48.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:48.650 INFO:teuthology.orchestra.run.smithi003.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:47:48.654 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/386 2023-11-04T06:47:48.654 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/386 2023-11-04T06:47:48.654 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:48.656 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:48.681 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:48.682 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2023-11-04T06:47:49.000 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:49.004 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:49.005 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:49.006 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:49.006 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:49.857 INFO:teuthology.orchestra.run.smithi003.stdout:generic/387 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:47:49.893 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/387 2023-11-04T06:47:49.893 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/387 2023-11-04T06:47:49.893 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:49.960 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:50.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:50.082 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2023-11-04T06:47:50.305 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:50.309 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:50.310 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:50.310 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:50.311 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:50.970 INFO:teuthology.orchestra.run.smithi003.stdout:generic/388 [not run] require 172.21.15.3:6789:/scratch to be local device 2023-11-04T06:47:50.971 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/388 2023-11-04T06:47:50.972 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/388 2023-11-04T06:47:50.972 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:50.974 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:51.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:51.007 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2023-11-04T06:47:51.239 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:51.243 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:51.243 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:51.244 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:51.598 INFO:teuthology.orchestra.run.smithi003.stdout:generic/389 [not run] O_TMPFILE is not supported 2023-11-04T06:47:51.602 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/389 2023-11-04T06:47:51.602 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/389 2023-11-04T06:47:51.603 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:51.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:51.628 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2023-11-04T06:47:51.866 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:51.871 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:51.871 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:51.872 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:52.273 INFO:teuthology.orchestra.run.smithi003.stdout:generic/390 [not run] ceph does not support freezing 2023-11-04T06:47:52.278 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/390 2023-11-04T06:47:52.278 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/390 2023-11-04T06:47:52.278 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:52.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:52.306 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2023-11-04T06:47:52.537 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:52.541 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:52.542 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:52.542 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:52.906 INFO:teuthology.orchestra.run.smithi003.stdout:generic/391 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:47:52.910 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/391 2023-11-04T06:47:52.911 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/391 2023-11-04T06:47:52.911 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:52.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:52.942 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2023-11-04T06:47:53.163 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:53.167 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:53.167 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:53.168 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:53.699 INFO:teuthology.orchestra.run.smithi003.stdout:generic/392 [not run] ceph does not support shutdown 2023-11-04T06:47:53.721 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/392 2023-11-04T06:47:53.722 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/392 2023-11-04T06:47:53.722 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:53.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:53.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:53.760 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2023-11-04T06:47:53.992 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:53.996 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:53.996 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:53.997 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:53.998 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:55.270 INFO:teuthology.orchestra.run.smithi003.stdout:generic/393 1s 2023-11-04T06:47:55.283 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/393 2023-11-04T06:47:55.283 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:55.285 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:55.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:55.295 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2023-11-04T06:47:55.526 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:55.531 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:55.531 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:55.532 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:55.532 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:55.917 INFO:teuthology.orchestra.run.smithi003.stdout:generic/394 0s 2023-11-04T06:47:55.922 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/394 2023-11-04T06:47:55.923 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:55.925 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:56.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:56.285 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2023-11-04T06:47:56.578 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:56.601 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:56.602 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:56.602 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:56.602 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:57.527 INFO:teuthology.orchestra.run.smithi003.stdout:generic/395 [not run] kernel does not support ceph encryption 2023-11-04T06:47:57.554 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/395 2023-11-04T06:47:57.555 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/395 2023-11-04T06:47:57.555 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:57.602 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:57.603 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2023-11-04T06:47:58.060 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:58.067 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:58.068 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:58.069 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:58.817 INFO:teuthology.orchestra.run.smithi003.stdout:generic/396 [not run] kernel does not support ceph encryption 2023-11-04T06:47:58.841 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/396 2023-11-04T06:47:58.841 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/396 2023-11-04T06:47:58.841 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:58.868 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:58.868 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2023-11-04T06:47:59.099 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:59.103 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:59.103 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:59.104 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:59.704 INFO:teuthology.orchestra.run.smithi003.stdout:generic/397 [not run] kernel does not support ceph encryption 2023-11-04T06:47:59.725 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/397 2023-11-04T06:47:59.726 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/397 2023-11-04T06:47:59.726 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:47:59.727 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:47:59.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:47:59.758 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2023-11-04T06:47:59.983 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:47:59.987 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:47:59.988 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:47:59.989 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:47:59.989 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:00.569 INFO:teuthology.orchestra.run.smithi003.stdout:generic/398 [not run] kernel does not support ceph encryption 2023-11-04T06:48:00.595 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/398 2023-11-04T06:48:00.595 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/398 2023-11-04T06:48:00.595 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:00.597 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:00.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:00.620 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2023-11-04T06:48:00.860 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:00.864 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:00.865 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:00.865 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:00.866 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:01.455 INFO:teuthology.orchestra.run.smithi003.stdout:generic/399 [not run] kernel does not support ceph encryption 2023-11-04T06:48:01.480 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/399 2023-11-04T06:48:01.480 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/399 2023-11-04T06:48:01.480 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:01.482 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:01.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:01.514 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2023-11-04T06:48:01.748 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:01.752 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:01.753 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:01.753 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:01.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:02.094 INFO:teuthology.orchestra.run.smithi003.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T06:48:02.098 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/400 2023-11-04T06:48:02.099 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/400 2023-11-04T06:48:02.099 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:02.100 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:02.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:02.125 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2023-11-04T06:48:02.354 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:02.358 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:02.359 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:02.360 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:02.360 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:generic/401 1s 2023-11-04T06:48:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/401 2023-11-04T06:48:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:03.401 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:03.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:03.411 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2023-11-04T06:48:03.645 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:03.649 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:03.650 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:03.651 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:03.651 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:04.220 INFO:teuthology.orchestra.run.smithi003.stdout:generic/402 [not run] filesystem ceph timestamp bounds are unknown 2023-11-04T06:48:04.239 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/402 2023-11-04T06:48:04.239 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/402 2023-11-04T06:48:04.239 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:04.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:04.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:04.278 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2023-11-04T06:48:04.514 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:04.518 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:04.519 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:04.520 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:04.520 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:generic/403 1s 2023-11-04T06:48:05.283 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/403 2023-11-04T06:48:05.283 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:05.285 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:05.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:05.294 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2023-11-04T06:48:05.531 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:05.535 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:05.536 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:05.536 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:05.537 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:05.907 INFO:teuthology.orchestra.run.smithi003.stdout:generic/404 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:48:05.911 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/404 2023-11-04T06:48:05.912 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/404 2023-11-04T06:48:05.912 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:05.913 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:05.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:05.956 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2023-11-04T06:48:06.185 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:06.189 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:06.190 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:06.191 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:06.191 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:06.607 INFO:teuthology.orchestra.run.smithi003.stdout:generic/405 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:48:06.611 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/405 2023-11-04T06:48:06.611 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/405 2023-11-04T06:48:06.611 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:06.613 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:06.640 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:06.641 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2023-11-04T06:48:06.866 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:06.870 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:06.871 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:06.872 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:06.872 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:08.806 INFO:teuthology.orchestra.run.smithi003.stdout:generic/406 1s 2023-11-04T06:48:08.811 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/406 2023-11-04T06:48:08.811 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:08.813 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:08.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:08.822 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2023-11-04T06:48:09.056 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:09.060 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:09.060 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:09.061 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:09.464 INFO:teuthology.orchestra.run.smithi003.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:48:09.468 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/407 2023-11-04T06:48:09.468 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/407 2023-11-04T06:48:09.469 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:09.470 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:09.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:09.794 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2023-11-04T06:48:10.227 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:10.231 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:10.231 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:10.232 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:10.614 INFO:teuthology.orchestra.run.smithi003.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T06:48:10.618 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/408 2023-11-04T06:48:10.618 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/408 2023-11-04T06:48:10.619 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:10.620 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:11.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:11.006 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2023-11-04T06:48:11.274 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:11.278 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:11.279 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:11.280 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:11.280 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:11.634 INFO:teuthology.orchestra.run.smithi003.stdout:generic/409 [not run] require 172.21.15.3:6789:/scratch to be local device 2023-11-04T06:48:11.639 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/409 2023-11-04T06:48:11.639 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/409 2023-11-04T06:48:11.639 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:11.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:11.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:11.665 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2023-11-04T06:48:11.892 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:11.896 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:11.897 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:11.898 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:11.898 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:12.253 INFO:teuthology.orchestra.run.smithi003.stdout:generic/410 [not run] require 172.21.15.3:6789:/scratch to be local device 2023-11-04T06:48:12.257 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/410 2023-11-04T06:48:12.258 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/410 2023-11-04T06:48:12.258 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:12.259 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:12.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:12.288 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2023-11-04T06:48:12.514 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:12.518 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:12.519 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:12.520 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:12.520 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:generic/411 [not run] require 172.21.15.3:6789:/scratch to be local device 2023-11-04T06:48:12.864 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/411 2023-11-04T06:48:12.865 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/411 2023-11-04T06:48:12.865 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:12.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:12.893 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2023-11-04T06:48:13.123 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:13.128 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:13.128 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:13.129 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:13.129 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:13.938 INFO:teuthology.orchestra.run.smithi003.stdout:generic/412 0s 2023-11-04T06:48:13.943 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/412 2023-11-04T06:48:13.943 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:13.945 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:13.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:13.954 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2023-11-04T06:48:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:14.193 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:14.681 INFO:teuthology.orchestra.run.smithi003.stdout:generic/413 [not run] mount 172.21.15.3:6789:/scratch with dax failed 2023-11-04T06:48:14.685 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/413 2023-11-04T06:48:14.686 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/413 2023-11-04T06:48:14.686 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:14.687 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:14.720 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:14.721 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2023-11-04T06:48:14.939 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:14.943 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:14.943 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:14.944 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:14.944 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:15.909 INFO:teuthology.orchestra.run.smithi003.stdout:generic/414 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:48:15.932 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/414 2023-11-04T06:48:15.933 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/414 2023-11-04T06:48:15.933 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:15.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:15.971 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2023-11-04T06:48:16.202 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:16.206 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:16.207 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:16.207 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:17.089 INFO:teuthology.orchestra.run.smithi003.stdout:generic/415 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:48:17.146 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/415 2023-11-04T06:48:17.146 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/415 2023-11-04T06:48:17.147 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:17.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:17.212 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2023-11-04T06:48:17.439 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:17.444 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:17.445 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:17.446 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:17.446 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:17.829 INFO:teuthology.orchestra.run.smithi003.stdout:generic/416 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:48:17.833 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/416 2023-11-04T06:48:17.833 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/416 2023-11-04T06:48:17.834 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:17.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:17.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:17.870 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2023-11-04T06:48:18.100 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:18.104 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:18.105 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:18.106 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:18.106 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:18.640 INFO:teuthology.orchestra.run.smithi003.stdout:generic/417 [not run] ceph does not support shutdown 2023-11-04T06:48:18.665 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/417 2023-11-04T06:48:18.666 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/417 2023-11-04T06:48:18.666 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:18.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:18.692 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2023-11-04T06:48:18.918 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:18.922 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:18.923 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:18.924 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:19.307 INFO:teuthology.orchestra.run.smithi003.stdout:generic/418 [not run] require 172.21.15.3:6789:/test to be valid block disk 2023-11-04T06:48:19.311 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/418 2023-11-04T06:48:19.312 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/418 2023-11-04T06:48:19.312 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:19.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:19.359 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2023-11-04T06:48:19.584 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:19.589 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:19.589 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:19.590 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:20.155 INFO:teuthology.orchestra.run.smithi003.stdout:generic/419 [not run] kernel does not support ceph encryption 2023-11-04T06:48:20.175 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/419 2023-11-04T06:48:20.175 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/419 2023-11-04T06:48:20.176 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:20.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:20.204 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2023-11-04T06:48:20.432 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:20.436 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:20.436 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:20.437 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:20.936 INFO:teuthology.orchestra.run.smithi003.stdout:generic/420 0s 2023-11-04T06:48:20.941 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/420 2023-11-04T06:48:20.941 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:21.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:21.376 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2023-11-04T06:48:21.610 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:21.614 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:21.615 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:21.616 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:21.616 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:22.203 INFO:teuthology.orchestra.run.smithi003.stdout:generic/421 [not run] kernel does not support ceph encryption 2023-11-04T06:48:22.227 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/421 2023-11-04T06:48:22.227 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/421 2023-11-04T06:48:22.228 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:22.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:22.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:22.263 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2023-11-04T06:48:22.488 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:22.493 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:22.493 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:22.494 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:22.494 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:generic/422 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T06:48:22.865 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/422 2023-11-04T06:48:22.866 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/422 2023-11-04T06:48:22.866 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:22.868 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:22.896 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:22.897 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2023-11-04T06:48:23.121 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:23.125 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:23.126 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:23.126 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:23.565 INFO:teuthology.orchestra.run.smithi003.stdout:generic/423 0s 2023-11-04T06:48:23.570 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/423 2023-11-04T06:48:23.570 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:23.572 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:23.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:23.978 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2023-11-04T06:48:24.257 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:24.261 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:24.262 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:24.263 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:24.626 INFO:teuthology.orchestra.run.smithi003.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2023-11-04T06:48:24.631 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/424 2023-11-04T06:48:24.631 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/424 2023-11-04T06:48:24.631 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:24.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:24.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:24.663 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2023-11-04T06:48:24.884 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:24.888 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:24.889 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:24.890 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:24.891 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:25.271 INFO:teuthology.orchestra.run.smithi003.stdout:generic/425 [not run] xfs_io fiemap -a failed (old kernel/wrong fs?) 2023-11-04T06:48:25.275 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/425 2023-11-04T06:48:25.275 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/425 2023-11-04T06:48:25.275 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:48:25.277 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:48:25.434 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:48:25.435 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2023-11-04T06:48:25.924 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:48:25.928 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:48:25.929 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:48:25.929 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:48:25.930 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:13.972 INFO:teuthology.orchestra.run.smithi003.stdout:generic/426 47s 2023-11-04T06:49:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/426 2023-11-04T06:49:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:14.082 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:14.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:14.517 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2023-11-04T06:49:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:15.509 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:16.003 INFO:teuthology.orchestra.run.smithi003.stdout:generic/427 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:49:16.007 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/427 2023-11-04T06:49:16.008 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/427 2023-11-04T06:49:16.008 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:16.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:16.039 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2023-11-04T06:49:16.271 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:16.275 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:16.276 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:16.276 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:16.662 INFO:teuthology.orchestra.run.smithi003.stdout:generic/428 0s 2023-11-04T06:49:16.668 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/428 2023-11-04T06:49:16.668 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:16.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:16.710 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2023-11-04T06:49:16.941 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:16.945 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:16.945 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:16.946 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:17.511 INFO:teuthology.orchestra.run.smithi003.stdout:generic/429 [not run] kernel does not support ceph encryption 2023-11-04T06:49:17.538 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/429 2023-11-04T06:49:17.538 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/429 2023-11-04T06:49:17.538 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:17.540 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:17.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:17.571 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2023-11-04T06:49:17.795 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:17.799 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:17.800 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:17.801 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:17.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:18.246 INFO:teuthology.orchestra.run.smithi003.stdout:generic/430 0s 2023-11-04T06:49:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/430 2023-11-04T06:49:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:18.254 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:18.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:18.647 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2023-11-04T06:49:18.898 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:18.903 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:18.903 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:18.904 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:19.310 INFO:teuthology.orchestra.run.smithi003.stdout:generic/431 0s 2023-11-04T06:49:19.315 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/431 2023-11-04T06:49:19.315 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:19.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:19.523 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2023-11-04T06:49:19.941 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:19.945 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:19.945 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:19.946 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:20.372 INFO:teuthology.orchestra.run.smithi003.stdout:generic/432 0s 2023-11-04T06:49:20.377 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/432 2023-11-04T06:49:20.378 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:20.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:20.781 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2023-11-04T06:49:21.010 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:21.014 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:21.015 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:21.016 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:21.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:21.414 INFO:teuthology.orchestra.run.smithi003.stdout:generic/433 0s 2023-11-04T06:49:21.418 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/433 2023-11-04T06:49:21.419 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:21.421 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:21.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:21.809 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2023-11-04T06:49:22.045 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:22.049 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:22.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:22.462 INFO:teuthology.orchestra.run.smithi003.stdout:generic/434 0s 2023-11-04T06:49:22.466 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/434 2023-11-04T06:49:22.467 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:22.468 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:22.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:22.861 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2023-11-04T06:49:23.097 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:23.101 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:23.101 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:23.102 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:23.659 INFO:teuthology.orchestra.run.smithi003.stdout:generic/435 [not run] kernel does not support ceph encryption 2023-11-04T06:49:23.680 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/435 2023-11-04T06:49:23.680 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/435 2023-11-04T06:49:23.680 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:23.682 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:23.706 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:23.707 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2023-11-04T06:49:23.929 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:23.933 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:23.934 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:23.934 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:24.586 INFO:teuthology.orchestra.run.smithi003.stdout:generic/436 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:49:24.590 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/436 2023-11-04T06:49:24.591 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/436 2023-11-04T06:49:24.591 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:24.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:24.939 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:24.940 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2023-11-04T06:49:25.217 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:25.221 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:25.221 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:25.222 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:25.223 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:26.221 INFO:teuthology.orchestra.run.smithi003.stdout:generic/437 1s 2023-11-04T06:49:26.226 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/437 2023-11-04T06:49:26.226 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:49:26.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:26.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:49:26.259 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2023-11-04T06:49:26.490 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:49:26.494 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:49:26.495 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:49:26.496 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:49:26.496 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:49:31.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:49:31.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:50:01.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:50:01.855 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:50:32.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:50:32.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:50:38.539 INFO:teuthology.orchestra.run.smithi003.stdout:generic/438 72s 2023-11-04T06:50:38.544 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/438 2023-11-04T06:50:38.544 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:38.546 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:38.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:38.574 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2023-11-04T06:50:38.801 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:38.805 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:38.805 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:38.809 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:38.809 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:39.972 INFO:teuthology.orchestra.run.smithi003.stdout:generic/439 0s 2023-11-04T06:50:39.977 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/439 2023-11-04T06:50:39.977 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:39.979 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:40.010 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:40.010 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2023-11-04T06:50:40.234 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:40.238 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:40.239 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:40.240 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:40.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:40.812 INFO:teuthology.orchestra.run.smithi003.stdout:generic/440 [not run] kernel does not support ceph encryption 2023-11-04T06:50:40.832 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/440 2023-11-04T06:50:40.833 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/440 2023-11-04T06:50:40.833 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:40.834 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:40.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:40.865 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2023-11-04T06:50:41.090 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:41.094 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:41.095 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:41.096 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:41.096 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:41.590 INFO:teuthology.orchestra.run.smithi003.stdout:generic/441 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:50:41.594 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/441 2023-11-04T06:50:41.594 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/441 2023-11-04T06:50:41.595 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:41.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:41.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:41.625 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2023-11-04T06:50:41.850 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:41.854 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:41.854 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:41.855 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:41.856 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:42.223 INFO:teuthology.orchestra.run.smithi003.stdout:generic/442 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:50:42.227 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/442 2023-11-04T06:50:42.227 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/442 2023-11-04T06:50:42.227 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:42.229 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:42.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:42.257 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2023-11-04T06:50:42.482 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:42.487 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:42.487 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:42.488 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:42.488 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:42.871 INFO:teuthology.orchestra.run.smithi003.stdout:generic/443 0s 2023-11-04T06:50:42.877 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/443 2023-11-04T06:50:42.877 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:42.879 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:42.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:42.910 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2023-11-04T06:50:43.133 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:43.137 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:43.138 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:43.139 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:43.139 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:43.566 INFO:teuthology.orchestra.run.smithi003.stdout:generic/444 0s 2023-11-04T06:50:43.571 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/444 2023-11-04T06:50:43.572 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:43.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:43.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:43.606 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2023-11-04T06:50:43.828 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:43.832 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:43.833 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:43.834 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:43.834 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:44.497 INFO:teuthology.orchestra.run.smithi003.stdout:generic/445 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:50:44.501 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/445 2023-11-04T06:50:44.501 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/445 2023-11-04T06:50:44.501 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:50:44.503 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:50:44.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:50:44.816 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2023-11-04T06:50:45.119 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:50:45.123 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:50:45.124 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:50:45.125 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:50:45.125 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:02.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:51:02.083 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:51:05.710 INFO:teuthology.orchestra.run.smithi003.stdout:generic/446 20s 2023-11-04T06:51:05.719 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/446 2023-11-04T06:51:05.720 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:05.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:05.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:05.734 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2023-11-04T06:51:05.960 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:05.963 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:05.964 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:05.965 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:05.965 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:06.834 INFO:teuthology.orchestra.run.smithi003.stdout:generic/447 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:51:06.871 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/447 2023-11-04T06:51:06.872 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/447 2023-11-04T06:51:06.872 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:06.877 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:06.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:06.911 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2023-11-04T06:51:07.155 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:07.159 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:07.159 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:07.160 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:07.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:08.236 INFO:teuthology.orchestra.run.smithi003.stdout:generic/448 1s 2023-11-04T06:51:08.240 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/448 2023-11-04T06:51:08.241 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:08.471 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:08.472 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2023-11-04T06:51:08.868 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:08.872 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:08.873 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:08.874 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:08.874 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:09.259 INFO:teuthology.orchestra.run.smithi003.stdout:generic/449 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T06:51:09.263 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/449 2023-11-04T06:51:09.263 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/449 2023-11-04T06:51:09.263 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:09.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:09.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:09.591 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2023-11-04T06:51:09.883 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:09.887 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:09.888 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:09.889 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:09.889 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:10.686 INFO:teuthology.orchestra.run.smithi003.stdout:generic/450 0s 2023-11-04T06:51:10.690 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/450 2023-11-04T06:51:10.691 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:10.693 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:11.040 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:11.041 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2023-11-04T06:51:11.297 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:11.301 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:11.301 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:11.302 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:11.302 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:32.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:51:32.170 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:51:41.805 INFO:teuthology.orchestra.run.smithi003.stdout:generic/451 30s 2023-11-04T06:51:41.810 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/451 2023-11-04T06:51:41.810 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:41.812 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:42.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:42.093 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2023-11-04T06:51:42.410 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:42.414 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:42.414 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:42.415 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:42.416 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:43.064 INFO:teuthology.orchestra.run.smithi003.stdout:generic/452 1s 2023-11-04T06:51:43.069 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/452 2023-11-04T06:51:43.070 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:43.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:43.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:43.081 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2023-11-04T06:51:43.308 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:43.312 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:43.313 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:43.314 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:43.314 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:43.661 INFO:teuthology.orchestra.run.smithi003.stdout:generic/453 [not run] ceph does not allow unrestricted byte streams for names 2023-11-04T06:51:43.665 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/453 2023-11-04T06:51:43.665 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/453 2023-11-04T06:51:43.665 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:43.667 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:43.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:43.689 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2023-11-04T06:51:43.918 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:43.922 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:43.923 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:43.924 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:43.924 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:44.271 INFO:teuthology.orchestra.run.smithi003.stdout:generic/454 [not run] ceph does not allow unrestricted byte streams for names 2023-11-04T06:51:44.275 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/454 2023-11-04T06:51:44.275 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/454 2023-11-04T06:51:44.276 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:44.278 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:44.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:44.318 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2023-11-04T06:51:44.546 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:44.550 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:44.550 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:44.551 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:44.551 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:44.972 INFO:teuthology.orchestra.run.smithi003.stdout:generic/455 [not run] This test requires a valid $LOGWRITES_DEV 2023-11-04T06:51:44.976 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/455 2023-11-04T06:51:44.977 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/455 2023-11-04T06:51:44.977 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:44.979 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:45.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:45.006 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2023-11-04T06:51:45.231 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:45.235 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:45.236 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:45.237 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:45.237 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:generic/456 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T06:51:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/456 2023-11-04T06:51:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/456 2023-11-04T06:51:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:45.610 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:45.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:45.647 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2023-11-04T06:51:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:45.879 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:46.830 INFO:teuthology.orchestra.run.smithi003.stdout:generic/457 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:51:46.836 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/457 2023-11-04T06:51:46.836 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/457 2023-11-04T06:51:46.836 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:46.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:46.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:46.866 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2023-11-04T06:51:47.089 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:47.093 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:47.094 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:47.095 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:47.095 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:47.994 INFO:teuthology.orchestra.run.smithi003.stdout:generic/458 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T06:51:48.005 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/458 2023-11-04T06:51:48.006 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/458 2023-11-04T06:51:48.006 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:48.009 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:48.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:48.047 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2023-11-04T06:51:48.286 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:48.290 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:48.290 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:48.291 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:48.291 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:48.829 INFO:teuthology.orchestra.run.smithi003.stdout:generic/459 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:51:48.833 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/459 2023-11-04T06:51:48.833 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/459 2023-11-04T06:51:48.833 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:48.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:48.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:48.868 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2023-11-04T06:51:49.089 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:49.093 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:49.094 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:49.095 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:49.095 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:51.013 INFO:teuthology.orchestra.run.smithi003.stdout:generic/460 2s 2023-11-04T06:51:51.018 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/460 2023-11-04T06:51:51.018 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:51.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:51.030 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2023-11-04T06:51:51.251 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:51.255 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:51.256 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:51.257 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:51.827 INFO:teuthology.orchestra.run.smithi003.stdout:generic/461 [not run] ceph does not support shutdown 2023-11-04T06:51:51.845 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/461 2023-11-04T06:51:51.845 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/461 2023-11-04T06:51:51.845 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:51.893 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:51.894 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2023-11-04T06:51:52.241 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:52.245 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:52.245 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:52.246 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:52.718 INFO:teuthology.orchestra.run.smithi003.stdout:generic/462 [not run] mount 172.21.15.3:6789:/scratch with dax failed 2023-11-04T06:51:52.722 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/462 2023-11-04T06:51:52.723 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/462 2023-11-04T06:51:52.723 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:52.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:52.756 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2023-11-04T06:51:52.979 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:52.983 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:52.983 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:52.984 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:53.365 INFO:teuthology.orchestra.run.smithi003.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph 2023-11-04T06:51:53.369 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/463 2023-11-04T06:51:53.369 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/463 2023-11-04T06:51:53.369 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:51:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:51:53.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:51:53.729 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2023-11-04T06:51:53.967 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:51:53.971 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:51:53.971 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:51:53.972 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:51:53.973 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:52:02.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:52:02.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:52:32.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:52:32.514 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:52:51.011 INFO:teuthology.orchestra.run.smithi003.stdout:generic/464 56s 2023-11-04T06:52:51.016 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/464 2023-11-04T06:52:51.016 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:52:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:52:51.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:52:51.028 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2023-11-04T06:52:51.251 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:52:51.255 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:52:51.256 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:52:51.257 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:52:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:52:55.911 INFO:teuthology.orchestra.run.smithi003.stdout:generic/465 4s 2023-11-04T06:52:55.919 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/465 2023-11-04T06:52:55.919 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:52:55.921 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:52:55.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:52:55.930 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2023-11-04T06:52:56.194 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:52:56.198 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:52:56.199 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:52:56.200 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:52:56.200 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:52:56.537 INFO:teuthology.orchestra.run.smithi003.stdout:generic/466 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:52:56.541 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/466 2023-11-04T06:52:56.542 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/466 2023-11-04T06:52:56.542 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:52:56.544 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:52:56.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:52:56.570 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2023-11-04T06:52:56.794 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:52:56.798 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:52:56.799 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:52:56.800 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:52:56.800 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:53:02.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:53:02.736 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:53:33.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:53:33.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:53:57.606 INFO:teuthology.orchestra.run.smithi003.stdout:generic/467 60s 2023-11-04T06:53:57.680 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/467 2023-11-04T06:53:57.681 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:53:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:53:58.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:53:58.125 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2023-11-04T06:53:59.076 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:53:59.080 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:53:59.081 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:53:59.082 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:53:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:53:59.697 INFO:teuthology.orchestra.run.smithi003.stdout:generic/468 [not run] ceph does not support shutdown 2023-11-04T06:53:59.714 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/468 2023-11-04T06:53:59.715 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/468 2023-11-04T06:53:59.715 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:53:59.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:53:59.745 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:53:59.746 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2023-11-04T06:53:59.978 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:53:59.982 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:53:59.982 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:53:59.983 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:53:59.983 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:00.340 INFO:teuthology.orchestra.run.smithi003.stdout:generic/469 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T06:54:00.344 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/469 2023-11-04T06:54:00.344 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/469 2023-11-04T06:54:00.344 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:00.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:00.380 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:00.381 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2023-11-04T06:54:00.602 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:00.606 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:00.606 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:00.607 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:00.607 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:00.997 INFO:teuthology.orchestra.run.smithi003.stdout:generic/470 [not run] This test requires a valid $LOGWRITES_DEV 2023-11-04T06:54:01.001 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/470 2023-11-04T06:54:01.001 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/470 2023-11-04T06:54:01.001 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:01.003 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:01.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:01.026 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2023-11-04T06:54:01.248 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:01.252 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:01.252 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:01.253 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:01.254 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:03.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:54:03.646 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:54:33.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:54:33.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:54:33.806 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T06:54:33.805+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119823) UID: 0 2023-11-04T06:54:33.806 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T06:54:33.805+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119823) UID: 0 2023-11-04T06:54:33.807 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T06:54:33.805+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119823) UID: 0 2023-11-04T06:54:33.807 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T06:54:33.805+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 119823) UID: 0 2023-11-04T06:54:46.279 INFO:teuthology.orchestra.run.smithi003.stdout:generic/471 45s 2023-11-04T06:54:46.285 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/471 2023-11-04T06:54:46.286 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:46.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:46.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:46.649 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2023-11-04T06:54:47.464 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:47.473 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:47.479 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:47.481 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:47.482 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:48.363 INFO:teuthology.orchestra.run.smithi003.stdout:generic/472 [not run] swapfiles are not supported 2023-11-04T06:54:48.367 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/472 2023-11-04T06:54:48.367 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/472 2023-11-04T06:54:48.367 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:48.369 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:48.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:48.400 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2023-11-04T06:54:48.624 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:48.628 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:48.629 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:48.630 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:48.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:49.019 INFO:teuthology.orchestra.run.smithi003.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-11-04T06:54:49.023 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/473 2023-11-04T06:54:49.023 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/473 2023-11-04T06:54:49.024 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:49.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:49.054 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2023-11-04T06:54:49.278 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:49.282 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:49.282 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:49.283 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:49.284 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:49.808 INFO:teuthology.orchestra.run.smithi003.stdout:generic/474 [not run] ceph does not support shutdown 2023-11-04T06:54:49.823 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/474 2023-11-04T06:54:49.823 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/474 2023-11-04T06:54:49.823 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:49.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:49.852 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2023-11-04T06:54:50.073 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:50.077 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:50.077 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:50.753 INFO:teuthology.orchestra.run.smithi003.stdout:generic/475 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T06:54:50.758 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/475 2023-11-04T06:54:50.758 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/475 2023-11-04T06:54:50.758 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T06:54:50.760 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:54:50.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T06:54:50.783 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2023-11-04T06:54:51.009 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T06:54:51.012 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T06:54:51.012 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T06:54:51.013 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T06:54:51.013 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T06:58:58.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:58:58.766 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T06:58:59.146 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T06:58:59.145+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614277) UID: 0 2023-11-04T06:58:59.146 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T06:58:59.145+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614277) UID: 0 2023-11-04T07:03:18.380 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T07:03:18.379+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614417) UID: 0 2023-11-04T07:03:18.381 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T07:03:18.379+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614417) UID: 0 2023-11-04T07:03:18.382 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T07:03:18.382+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614417) UID: 0 2023-11-04T07:03:18.382 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T07:03:18.379+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614417) UID: 0 2023-11-04T07:07:58.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:07:58.757 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:08:28.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:08:28.997 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:08:29.082 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T07:08:29.081+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120254) UID: 0 2023-11-04T07:08:29.082 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T07:08:29.082+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120254) UID: 0 2023-11-04T07:08:29.084 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T07:08:29.081+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120254) UID: 0 2023-11-04T07:08:29.085 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T07:08:29.081+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120254) UID: 0 2023-11-04T07:12:43.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:12:43.691 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:12:44.001 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T07:12:43.984+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614585) UID: 0 2023-11-04T07:12:44.001 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T07:12:43.984+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614585) UID: 0 2023-11-04T07:12:44.014 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T07:12:43.989+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614585) UID: 0 2023-11-04T07:12:44.017 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T07:12:43.984+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614585) UID: 0 2023-11-04T07:20:46.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:20:46.327 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:20:46.696 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T07:20:46.693+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120686) UID: 0 2023-11-04T07:20:46.697 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T07:20:46.693+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120686) UID: 0 2023-11-04T07:20:46.698 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T07:20:46.693+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120686) UID: 0 2023-11-04T07:20:46.702 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T07:20:46.699+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120686) UID: 0 2023-11-04T07:20:47.181 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T07:20:47.177+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614694) UID: 0 2023-11-04T07:20:47.182 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T07:20:47.177+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614694) UID: 0 2023-11-04T07:25:58.902 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T07:25:58.900+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614794) UID: 0 2023-11-04T07:25:58.903 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T07:25:58.900+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614794) UID: 0 2023-11-04T07:25:58.903 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T07:25:58.901+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614794) UID: 0 2023-11-04T07:25:58.907 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T07:25:58.906+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 614794) UID: 0 2023-11-04T07:31:07.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:31:08.003 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:31:08.359 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T07:31:08.357+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120831) UID: 0 2023-11-04T07:31:08.361 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T07:31:08.361+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120831) UID: 0 2023-11-04T07:31:08.365 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T07:31:08.361+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120831) UID: 0 2023-11-04T07:31:08.366 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T07:31:08.361+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120831) UID: 0 2023-11-04T07:36:15.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:36:15.958 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:36:16.385 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T07:36:16.383+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614956) UID: 0 2023-11-04T07:36:16.388 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T07:36:16.387+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614956) UID: 0 2023-11-04T07:40:39.990 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T07:40:39.989+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615006) UID: 0 2023-11-04T07:40:39.992 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T07:40:39.989+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615006) UID: 0 2023-11-04T07:40:39.992 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T07:40:39.990+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615006) UID: 0 2023-11-04T07:40:39.993 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T07:40:39.989+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615006) UID: 0 2023-11-04T07:45:25.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:45:25.129 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:45:25.234 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T07:45:25.234+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121125) UID: 0 2023-11-04T07:45:25.238 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T07:45:25.238+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121125) UID: 0 2023-11-04T07:45:25.239 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T07:45:25.237+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121125) UID: 0 2023-11-04T07:45:25.240 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T07:45:25.238+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121125) UID: 0 2023-11-04T07:45:26.508 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T07:45:26.508+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615084) UID: 0 2023-11-04T07:45:26.509 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T07:45:26.508+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615084) UID: 0 2023-11-04T07:45:26.518 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T07:45:26.508+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615084) UID: 0 2023-11-04T07:45:26.519 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T07:45:26.506+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615084) UID: 0 2023-11-04T07:50:04.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:50:04.152 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:50:04.338 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T07:50:04.337+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121198) UID: 0 2023-11-04T07:50:04.339 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T07:50:04.337+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121198) UID: 0 2023-11-04T07:50:04.339 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T07:50:04.337+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121198) UID: 0 2023-11-04T07:50:04.341 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T07:50:04.341+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121198) UID: 0 2023-11-04T07:50:04.735 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T07:50:04.734+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 615152) UID: 0 2023-11-04T07:50:04.739 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T07:50:04.734+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 615152) UID: 0 2023-11-04T07:53:51.556 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T07:53:51.554+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615194) UID: 0 2023-11-04T07:53:51.556 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T07:53:51.554+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615194) UID: 0 2023-11-04T07:53:51.557 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T07:53:51.554+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615194) UID: 0 2023-11-04T07:53:51.557 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T07:53:51.555+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 615194) UID: 0 2023-11-04T07:58:05.311 INFO:teuthology.orchestra.run.smithi003.stdout:generic/476 3794s 2023-11-04T07:58:05.350 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/476 2023-11-04T07:58:05.351 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T07:58:05.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T07:58:05.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T07:58:05.365 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2023-11-04T07:58:19.852 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T07:58:19.856 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T07:58:19.858 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T07:58:19.859 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T07:58:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T07:58:25.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:58:25.096 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:58:55.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:58:55.240 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:59:25.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:59:25.325 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:59:55.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:59:55.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T07:59:55.757 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T07:59:55.756+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121375) UID: 0 2023-11-04T07:59:55.758 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T07:59:55.756+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121375) UID: 0 2023-11-04T07:59:55.758 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T07:59:55.756+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121375) UID: 0 2023-11-04T07:59:55.759 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T07:59:55.756+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121375) UID: 0 2023-11-04T07:59:55.772 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T07:59:55.770+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 615631) UID: 0 2023-11-04T07:59:55.772 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T07:59:55.770+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 615631) UID: 0 2023-11-04T08:03:16.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:03:16.642 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:03:16.738 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:03:16.737+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121447) UID: 0 2023-11-04T08:03:16.739 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:03:16.738+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 121447) UID: 0 2023-11-04T08:03:16.744 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:03:16.737+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121447) UID: 0 2023-11-04T08:03:16.744 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:03:16.738+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 121447) UID: 0 2023-11-04T08:06:05.052 INFO:teuthology.orchestra.run.smithi003.stdout:generic/477 30s 2023-11-04T08:06:05.117 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/477 2023-11-04T08:06:05.117 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:05.143 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:05.266 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:05.267 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2023-11-04T08:06:06.425 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:06.431 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:06.432 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:06.433 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:17.316 INFO:teuthology.orchestra.run.smithi003.stdout:generic/478 11s 2023-11-04T08:06:17.322 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/478 2023-11-04T08:06:17.323 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:17.325 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:17.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:17.383 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2023-11-04T08:06:17.762 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:17.776 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:17.776 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:17.777 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:17.777 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:18.416 INFO:teuthology.orchestra.run.smithi003.stdout:generic/479 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:18.421 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/479 2023-11-04T08:06:18.421 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/479 2023-11-04T08:06:18.421 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:18.423 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:18.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:18.473 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2023-11-04T08:06:18.864 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:18.874 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:18.875 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:18.876 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:19.499 INFO:teuthology.orchestra.run.smithi003.stdout:generic/480 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:19.507 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/480 2023-11-04T08:06:19.508 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/480 2023-11-04T08:06:19.508 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:19.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:19.548 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2023-11-04T08:06:19.938 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:19.944 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:19.945 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:19.946 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:20.446 INFO:teuthology.orchestra.run.smithi003.stdout:generic/481 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:20.451 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/481 2023-11-04T08:06:20.451 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/481 2023-11-04T08:06:20.451 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:20.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:20.490 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2023-11-04T08:06:20.816 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:20.823 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:20.826 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:20.827 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:21.422 INFO:teuthology.orchestra.run.smithi003.stdout:generic/482 [not run] This test requires a valid $LOGWRITES_DEV 2023-11-04T08:06:21.427 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/482 2023-11-04T08:06:21.427 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/482 2023-11-04T08:06:21.427 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:21.433 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:21.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:21.459 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2023-11-04T08:06:21.832 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:21.837 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:21.837 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:21.838 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:21.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:22.357 INFO:teuthology.orchestra.run.smithi003.stdout:generic/483 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:22.361 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/483 2023-11-04T08:06:22.361 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/483 2023-11-04T08:06:22.362 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:22.363 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:22.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:22.393 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2023-11-04T08:06:22.783 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:22.796 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:22.799 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:22.800 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:22.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:23.313 INFO:teuthology.orchestra.run.smithi003.stdout:generic/484 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:23.318 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/484 2023-11-04T08:06:23.319 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/484 2023-11-04T08:06:23.319 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:23.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:23.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:23.357 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2023-11-04T08:06:23.720 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:23.724 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:23.725 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:23.726 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:24.271 INFO:teuthology.orchestra.run.smithi003.stdout:generic/485 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T08:06:24.279 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/485 2023-11-04T08:06:24.279 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/485 2023-11-04T08:06:24.279 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:24.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:24.323 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2023-11-04T08:06:24.683 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:24.691 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:24.692 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:24.693 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:24.694 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:25.611 INFO:teuthology.orchestra.run.smithi003.stdout:generic/486 1s 2023-11-04T08:06:25.620 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/486 2023-11-04T08:06:25.620 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:25.625 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:25.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:25.636 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2023-11-04T08:06:26.012 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:26.016 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:26.016 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:26.017 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:26.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:26.552 INFO:teuthology.orchestra.run.smithi003.stdout:generic/487 [not run] This test requires a valid $SCRATCH_LOGDEV 2023-11-04T08:06:26.558 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/487 2023-11-04T08:06:26.559 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/487 2023-11-04T08:06:26.559 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:26.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:26.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:26.590 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2023-11-04T08:06:26.933 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:26.942 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:26.942 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:26.944 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:26.944 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:27.433 INFO:teuthology.orchestra.run.smithi003.stdout:generic/488 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-11-04T08:06:27.438 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/488 2023-11-04T08:06:27.438 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/488 2023-11-04T08:06:27.439 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:27.440 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:27.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:27.467 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2023-11-04T08:06:27.870 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:27.883 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:27.883 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:27.885 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:27.885 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:generic/489 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/489 2023-11-04T08:06:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/489 2023-11-04T08:06:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:28.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:28.489 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:28.490 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2023-11-04T08:06:28.837 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:28.847 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:28.847 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:28.848 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:28.848 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:31.152 INFO:teuthology.orchestra.run.smithi003.stdout:generic/490 2s 2023-11-04T08:06:31.168 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/490 2023-11-04T08:06:31.169 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:31.171 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:31.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:06:31.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:06:31.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:31.540 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2023-11-04T08:06:31.990 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:31.994 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:31.995 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:31.996 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:31.996 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:32.539 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T08:06:32.537+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 623571) UID: 0 2023-11-04T08:06:32.540 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T08:06:32.537+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 623571) UID: 0 2023-11-04T08:06:33.182 INFO:teuthology.orchestra.run.smithi003.stdout:generic/491 [not run] ceph does not support freezing 2023-11-04T08:06:33.189 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/491 2023-11-04T08:06:33.189 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/491 2023-11-04T08:06:33.189 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:33.199 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:33.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:33.227 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2023-11-04T08:06:33.686 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:33.696 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:33.696 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:33.701 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:33.701 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:generic/492 [not run] xfs_io label failed (old kernel/wrong fs?) 2023-11-04T08:06:34.349 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/492 2023-11-04T08:06:34.349 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/492 2023-11-04T08:06:34.349 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:34.355 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:34.408 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:34.409 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2023-11-04T08:06:34.891 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:34.911 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:34.911 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:34.912 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:34.912 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:36.068 INFO:teuthology.orchestra.run.smithi003.stdout:generic/493 [not run] swapfiles are not supported 2023-11-04T08:06:36.081 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/493 2023-11-04T08:06:36.081 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/493 2023-11-04T08:06:36.082 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:36.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:36.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:36.132 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2023-11-04T08:06:36.608 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:36.618 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:36.618 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:36.619 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:36.619 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:37.611 INFO:teuthology.orchestra.run.smithi003.stdout:generic/494 [not run] swapfiles are not supported 2023-11-04T08:06:37.617 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/494 2023-11-04T08:06:37.617 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/494 2023-11-04T08:06:37.618 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:37.624 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:37.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:37.665 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2023-11-04T08:06:38.154 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:38.160 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:38.160 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:38.161 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:38.162 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:39.192 INFO:teuthology.orchestra.run.smithi003.stdout:generic/495 [not run] swapfiles are not supported 2023-11-04T08:06:39.196 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/495 2023-11-04T08:06:39.196 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/495 2023-11-04T08:06:39.196 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:39.201 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:39.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:39.254 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2023-11-04T08:06:39.731 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:39.744 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:39.778 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:39.779 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:39.779 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:40.775 INFO:teuthology.orchestra.run.smithi003.stdout:generic/496 [not run] swapfiles are not supported 2023-11-04T08:06:40.786 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/496 2023-11-04T08:06:40.787 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/496 2023-11-04T08:06:40.787 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:40.791 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:40.836 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:40.837 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2023-11-04T08:06:41.269 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:41.274 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:41.275 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:41.276 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:41.276 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:42.333 INFO:teuthology.orchestra.run.smithi003.stdout:generic/497 [not run] swapfiles are not supported 2023-11-04T08:06:42.338 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/497 2023-11-04T08:06:42.338 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/497 2023-11-04T08:06:42.339 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:42.341 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:42.392 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:42.393 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2023-11-04T08:06:42.847 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:42.854 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:42.855 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:42.860 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:42.861 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:43.466 INFO:teuthology.orchestra.run.smithi003.stdout:generic/498 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:43.472 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/498 2023-11-04T08:06:43.472 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/498 2023-11-04T08:06:43.473 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:43.489 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:43.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:43.548 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2023-11-04T08:06:44.049 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:44.063 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:44.064 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:44.066 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:44.066 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:44.606 INFO:teuthology.orchestra.run.smithi003.stdout:generic/499 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T08:06:44.611 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/499 2023-11-04T08:06:44.611 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/499 2023-11-04T08:06:44.612 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:44.615 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:44.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:44.672 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2023-11-04T08:06:45.114 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:45.126 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:45.126 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:45.127 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:45.127 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:generic/500 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/500 2023-11-04T08:06:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/500 2023-11-04T08:06:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:45.756 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:45.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:45.798 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2023-11-04T08:06:46.275 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:46.283 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:46.284 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:46.288 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:46.289 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:47.693 INFO:teuthology.orchestra.run.smithi003.stdout:generic/501 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T08:06:47.733 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/501 2023-11-04T08:06:47.734 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/501 2023-11-04T08:06:47.734 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:47.737 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:47.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:47.786 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2023-11-04T08:06:48.256 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:48.262 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:48.262 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:48.263 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:48.264 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:48.988 INFO:teuthology.orchestra.run.smithi003.stdout:generic/502 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/502 2023-11-04T08:06:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/502 2023-11-04T08:06:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:48.995 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:49.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:49.066 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2023-11-04T08:06:49.520 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:49.525 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:49.525 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:49.526 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:49.526 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:50.177 INFO:teuthology.orchestra.run.smithi003.stdout:generic/503 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T08:06:50.186 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/503 2023-11-04T08:06:50.187 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/503 2023-11-04T08:06:50.187 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:50.189 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:50.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:50.246 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2023-11-04T08:06:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:50.643 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:50.645 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:50.647 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:50.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:51.308 INFO:teuthology.orchestra.run.smithi003.stdout:generic/504 1s 2023-11-04T08:06:51.314 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/504 2023-11-04T08:06:51.315 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:51.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:51.370 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2023-11-04T08:06:51.820 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:51.831 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:51.832 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:51.835 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:52.704 INFO:teuthology.orchestra.run.smithi003.stdout:generic/505 [not run] ceph does not support shutdown 2023-11-04T08:06:52.726 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/505 2023-11-04T08:06:52.726 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/505 2023-11-04T08:06:52.726 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:52.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:52.774 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2023-11-04T08:06:53.171 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:53.176 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:53.177 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:53.178 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:53.794 INFO:teuthology.orchestra.run.smithi003.stdout:generic/506 [not run] disk quotas not supported by this filesystem type: ceph 2023-11-04T08:06:53.799 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/506 2023-11-04T08:06:53.799 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/506 2023-11-04T08:06:53.799 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:53.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:53.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:53.843 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2023-11-04T08:06:54.333 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:54.337 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:54.338 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:54.339 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:54.943 INFO:teuthology.orchestra.run.smithi003.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2023-11-04T08:06:54.955 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/507 2023-11-04T08:06:54.955 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/507 2023-11-04T08:06:54.955 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:54.960 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:55.007 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:55.008 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2023-11-04T08:06:55.452 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:55.459 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:55.460 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:55.464 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:55.465 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:56.065 INFO:teuthology.orchestra.run.smithi003.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph 2023-11-04T08:06:56.070 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/508 2023-11-04T08:06:56.071 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/508 2023-11-04T08:06:56.071 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:56.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:56.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:56.111 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2023-11-04T08:06:56.601 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:56.607 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:56.607 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:56.608 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:56.608 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:57.261 INFO:teuthology.orchestra.run.smithi003.stdout:generic/509 [not run] O_TMPFILE is not supported 2023-11-04T08:06:57.273 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/509 2023-11-04T08:06:57.274 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/509 2023-11-04T08:06:57.274 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:57.281 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:57.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:57.316 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2023-11-04T08:06:57.745 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:57.751 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:57.751 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:57.753 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:58.355 INFO:teuthology.orchestra.run.smithi003.stdout:generic/510 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:06:58.356 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/510 2023-11-04T08:06:58.357 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/510 2023-11-04T08:06:58.357 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:58.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:58.402 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2023-11-04T08:06:58.868 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:59.465 INFO:teuthology.orchestra.run.smithi003.stdout:generic/511 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-11-04T08:06:59.470 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/511 2023-11-04T08:06:59.470 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/511 2023-11-04T08:06:59.470 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:06:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:06:59.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:06:59.520 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2023-11-04T08:06:59.897 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:06:59.909 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:06:59.910 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:06:59.910 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:06:59.911 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:00.567 INFO:teuthology.orchestra.run.smithi003.stdout:generic/512 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T08:07:00.578 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/512 2023-11-04T08:07:00.579 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/512 2023-11-04T08:07:00.579 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:00.584 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:00.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:00.624 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2023-11-04T08:07:01.166 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:01.174 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:01.175 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:02.460 INFO:teuthology.orchestra.run.smithi003.stdout:generic/513 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T08:07:02.554 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/513 2023-11-04T08:07:02.554 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/513 2023-11-04T08:07:02.554 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:02.591 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:02.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:02.636 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2023-11-04T08:07:03.115 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:03.133 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:03.134 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:03.134 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:03.134 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:04.513 INFO:teuthology.orchestra.run.smithi003.stdout:generic/514 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T08:07:04.551 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/514 2023-11-04T08:07:04.551 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/514 2023-11-04T08:07:04.551 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:04.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:04.608 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:04.608 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2023-11-04T08:07:05.065 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:05.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:06.551 INFO:teuthology.orchestra.run.smithi003.stdout:generic/515 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T08:07:06.602 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/515 2023-11-04T08:07:06.603 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/515 2023-11-04T08:07:06.603 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:06.645 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:06.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:06.691 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2023-11-04T08:07:07.079 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:07.090 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:07.096 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:07.098 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:07.098 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph 2023-11-04T08:07:07.791 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/516 2023-11-04T08:07:07.792 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/516 2023-11-04T08:07:07.792 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:07.794 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:08.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:08.243 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2023-11-04T08:07:08.764 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:08.772 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:08.773 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:08.774 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:08.774 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:09.675 INFO:teuthology.orchestra.run.smithi003.stdout:generic/517 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-11-04T08:07:09.988 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/517 2023-11-04T08:07:09.988 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/517 2023-11-04T08:07:09.989 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:10.082 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:10.188 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:10.189 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2023-11-04T08:07:10.606 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:10.622 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:10.622 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:10.623 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:10.624 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:11.920 INFO:teuthology.orchestra.run.smithi003.stdout:generic/518 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T08:07:12.034 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/518 2023-11-04T08:07:12.035 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/518 2023-11-04T08:07:12.035 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:12.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:12.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:12.125 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2023-11-04T08:07:12.564 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:12.569 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:12.570 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:12.571 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:12.571 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:13.197 INFO:teuthology.orchestra.run.smithi003.stdout:generic/519 [not run] FIBMAP not supported by this filesystem 2023-11-04T08:07:13.205 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/519 2023-11-04T08:07:13.206 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/519 2023-11-04T08:07:13.206 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:13.208 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:13.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:13.569 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2023-11-04T08:07:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:14.154 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:14.789 INFO:teuthology.orchestra.run.smithi003.stdout:generic/520 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T08:07:14.795 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/520 2023-11-04T08:07:14.796 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/520 2023-11-04T08:07:14.796 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T08:07:14.798 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:07:14.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T08:07:14.841 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2023-11-04T08:07:15.322 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T08:07:15.327 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T08:07:15.328 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T08:07:15.329 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T08:07:15.329 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T08:10:01.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:10:01.512 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:10:01.605 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f1f6ddbb700 -1 Fail to open '/proc/638201/cmdline' error = (2) No such file or directory 2023-11-04T08:10:01.606 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f1f6ddbb700 -1 received signal: Hangup from (PID: 638201) UID: 0 2023-11-04T08:10:01.606 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f7dba50c700 -1 Fail to open '/proc/638201/cmdline' error = (2) No such file or directory 2023-11-04T08:10:01.606 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f7dba50c700 -1 received signal: Hangup from (PID: 638201) UID: 0 2023-11-04T08:10:01.609 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f58ab082700 -1 Fail to open '/proc/638201/cmdline' error = (2) No such file or directory 2023-11-04T08:10:01.609 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f58ab082700 -1 received signal: Hangup from (PID: 638201) UID: 0 2023-11-04T08:10:01.609 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f97f5987700 -1 Fail to open '/proc/638201/cmdline' error = (2) No such file or directory 2023-11-04T08:10:01.610 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:10:01.603+0000 7f97f5987700 -1 received signal: Hangup from (PID: 638201) UID: 0 2023-11-04T08:13:22.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:13:22.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:13:53.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:13:53.094 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:14:23.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:14:23.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:14:53.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:14:53.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:15:23.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:15:23.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:15:53.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:15:53.873 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:16:24.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:16:24.186 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:16:54.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:16:54.262 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:16:54.312 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:16:54.310+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638432) UID: 0 2023-11-04T08:16:54.312 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:16:54.310+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638432) UID: 0 2023-11-04T08:16:54.313 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:16:54.310+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638432) UID: 0 2023-11-04T08:16:54.313 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:16:54.310+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638432) UID: 0 2023-11-04T08:20:17.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:20:17.386 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:20:17.431 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:20:17.429+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638495) UID: 0 2023-11-04T08:20:17.432 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:20:17.429+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638495) UID: 0 2023-11-04T08:20:17.432 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:20:17.429+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638495) UID: 0 2023-11-04T08:20:17.433 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:20:17.429+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 638495) UID: 0 2023-11-04T08:20:17.476 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:20:17.474+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121830) UID: 0 2023-11-04T08:20:17.477 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:20:17.474+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121830) UID: 0 2023-11-04T08:20:17.477 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:20:17.474+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121830) UID: 0 2023-11-04T08:20:17.478 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:20:17.474+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121830) UID: 0 2023-11-04T08:23:41.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:23:41.097 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:23:41.198 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:23:41.196+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121881) UID: 0 2023-11-04T08:23:41.199 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:23:41.196+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 121881) UID: 0 2023-11-04T08:23:41.199 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:23:41.197+0000 7f8a2900f700 -1 Fail to open '/proc/121881/cmdline' error = (2) No such file or directory 2023-11-04T08:23:41.200 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:23:41.197+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 121881) UID: 0 2023-11-04T08:23:41.200 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:23:41.197+0000 7fdadcd9d700 -1 Fail to open '/proc/121881/cmdline' error = (2) No such file or directory 2023-11-04T08:23:41.200 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:23:41.197+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 121881) UID: 0 2023-11-04T08:27:15.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:27:15.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:27:45.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:27:45.658 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:28:15.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:28:15.751 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:28:45.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:28:45.894 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:29:15.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:29:15.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:29:46.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:29:46.084 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:30:16.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:30:16.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:30:46.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:30:46.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:31:16.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:31:16.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:31:47.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:31:47.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:32:17.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:32:17.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:32:47.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:32:47.314 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:33:17.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:33:17.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:33:47.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:33:47.526 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:34:17.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:34:17.849 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:34:48.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:34:48.039 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:35:18.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:35:18.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:35:48.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:35:48.229 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:36:18.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:36:18.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:36:48.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:36:48.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:37:18.546 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:37:18.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:37:48.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:37:48.703 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:38:18.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:38:18.879 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:38:49.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:38:49.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:39:19.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:39:19.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:39:49.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:39:49.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:40:19.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:40:19.835 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:40:19.884 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:40:19.882+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122600) UID: 0 2023-11-04T08:40:19.885 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:40:19.882+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122600) UID: 0 2023-11-04T08:40:19.885 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:40:19.882+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 122600) UID: 0 2023-11-04T08:40:19.886 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:40:19.882+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122600) UID: 0 2023-11-04T08:40:19.917 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:40:19.916+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639261) UID: 0 2023-11-04T08:40:19.918 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:40:19.916+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639261) UID: 0 2023-11-04T08:40:19.918 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:40:19.916+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639261) UID: 0 2023-11-04T08:40:19.919 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:40:19.918+0000 7f58ab082700 -1 Fail to open '/proc/639261/cmdline' error = (2) No such file or directory 2023-11-04T08:40:19.919 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:40:19.918+0000 7f58ab082700 -1 received signal: Hangup from (PID: 639261) UID: 0 2023-11-04T08:43:38.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:43:38.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:44:08.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:44:08.374 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:44:38.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:44:38.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:45:08.606 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:45:08.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:45:38.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:45:38.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:46:08.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:46:08.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:46:08.893 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:46:08.892+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122783) UID: 0 2023-11-04T08:46:08.894 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:46:08.892+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122783) UID: 0 2023-11-04T08:46:08.894 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:46:08.892+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122783) UID: 0 2023-11-04T08:46:08.895 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:46:08.892+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122783) UID: 0 2023-11-04T08:49:31.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:49:31.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:49:31.692 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:49:31.690+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122833) UID: 0 2023-11-04T08:49:31.693 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:49:31.691+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122833) UID: 0 2023-11-04T08:49:31.693 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:49:31.691+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 122833) UID: 0 2023-11-04T08:49:31.694 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:49:31.691+0000 7fdadcd9d700 -1 Fail to open '/proc/122833/cmdline' error = (2) No such file or directory 2023-11-04T08:49:31.694 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:49:31.691+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 122833) UID: 0 2023-11-04T08:52:54.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:52:54.650 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:52:54.722 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:52:54.718+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639740) UID: 0 2023-11-04T08:52:54.723 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:52:54.720+0000 7f97f5987700 -1 received signal: Hangup from (PID: 639740) UID: 0 2023-11-04T08:52:54.723 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:52:54.720+0000 7f1f6ddbb700 -1 received signal: Hangup from (PID: 639740) UID: 0 2023-11-04T08:52:54.725 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:52:54.723+0000 7f58ab082700 -1 Fail to open '/proc/639740/cmdline' error = (2) No such file or directory 2023-11-04T08:52:54.726 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:52:54.723+0000 7f58ab082700 -1 received signal: Hangup from (PID: 639740) UID: 0 2023-11-04T08:56:31.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:56:31.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T08:56:31.274 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T08:56:31.272+0000 7eff274c1700 -1 received signal: Hangup from (PID: 122944) UID: 0 2023-11-04T08:56:31.274 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T08:56:31.269+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 122944) UID: 0 2023-11-04T08:56:31.275 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:56:31.273+0000 7fdadcd9d700 -1 Fail to open '/proc/122944/cmdline' error = (2) No such file or directory 2023-11-04T08:56:31.275 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T08:56:31.273+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 122944) UID: 0 2023-11-04T08:56:31.276 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T08:56:31.272+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639799) UID: 0 2023-11-04T08:56:31.276 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T08:56:31.272+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639799) UID: 0 2023-11-04T08:56:31.276 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T08:56:31.272+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639799) UID: 0 2023-11-04T08:56:31.277 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T08:56:31.272+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639799) UID: 0 2023-11-04T08:56:31.277 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T08:56:31.272+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 122944) UID: 0 2023-11-04T09:00:01.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:00:01.102 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:00:31.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:00:31.302 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:01:01.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:01:01.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:01:01.461 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T09:01:01.460+0000 7f1f6ddbb700 -1 received signal: Hangup from (PID: 639915) UID: 0 2023-11-04T09:01:01.461 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:01:01.458+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639915) UID: 0 2023-11-04T09:01:01.462 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:01:01.459+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 639915) UID: 0 2023-11-04T09:01:01.464 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:01:01.460+0000 7f58ab082700 -1 received signal: Hangup from (PID: 639915) UID: 0 2023-11-04T09:04:20.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:04:20.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:04:50.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:04:50.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:05:20.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:05:20.657 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:05:51.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:05:51.174 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:06:21.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:06:21.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:06:51.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:06:51.363 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:07:21.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:07:21.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:07:51.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:07:51.769 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:08:21.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:08:21.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:08:52.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:08:52.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:09:22.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:09:22.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:09:52.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:09:52.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:10:22.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:10:22.688 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:10:52.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:10:52.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:11:22.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:11:22.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:11:53.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:11:53.015 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:11:53.102 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:11:53.101+0000 7f8a2900f700 -1 Fail to open '/proc/123490/cmdline' error = (2) No such file or directory 2023-11-04T09:11:53.103 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:11:53.101+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 123490) UID: 0 2023-11-04T09:11:53.103 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:11:53.101+0000 7fdadcd9d700 -1 Fail to open '/proc/123490/cmdline' error = (2) No such file or directory 2023-11-04T09:11:53.103 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:11:53.101+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 123490) UID: 0 2023-11-04T09:11:53.104 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:11:53.102+0000 7f1044bc0700 -1 Fail to open '/proc/123490/cmdline' error = (2) No such file or directory 2023-11-04T09:11:53.104 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:11:53.102+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 123490) UID: 0 2023-11-04T09:11:53.104 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T09:11:53.099+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123490) UID: 0 2023-11-04T09:15:14.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:15:14.766 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:15:44.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:15:44.983 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:16:15.108 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:16:15.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:16:45.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:16:45.206 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:17:15.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:17:15.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:17:45.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:17:45.387 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:17:45.441 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T09:17:45.439+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123669) UID: 0 2023-11-04T09:17:45.441 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:17:45.439+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123669) UID: 0 2023-11-04T09:17:45.442 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:17:45.439+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123669) UID: 0 2023-11-04T09:17:45.442 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:17:45.439+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123669) UID: 0 2023-11-04T09:21:02.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:21:02.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:21:32.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:21:32.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:22:03.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:22:03.041 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:22:33.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:22:33.592 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:23:03.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:23:03.698 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:23:33.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:23:34.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:24:04.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:24:04.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:24:34.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:24:34.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:25:04.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:25:04.286 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:25:04.343 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T09:25:04.342+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640780) UID: 0 2023-11-04T09:25:04.347 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:25:04.347+0000 7f58ab082700 -1 Fail to open '/proc/640780/cmdline' error = (2) No such file or directory 2023-11-04T09:25:04.348 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:25:04.347+0000 7f58ab082700 -1 received signal: Hangup from (PID: 640780) UID: 0 2023-11-04T09:25:04.348 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:25:04.347+0000 7f7dba50c700 -1 Fail to open '/proc/640780/cmdline' error = (2) No such file or directory 2023-11-04T09:25:04.348 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:25:04.347+0000 7f7dba50c700 -1 received signal: Hangup from (PID: 640780) UID: 0 2023-11-04T09:25:04.349 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:25:04.347+0000 7f97f5987700 -1 Fail to open '/proc/640780/cmdline' error = (2) No such file or directory 2023-11-04T09:25:04.349 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:25:04.347+0000 7f97f5987700 -1 received signal: Hangup from (PID: 640780) UID: 0 2023-11-04T09:28:28.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:28:28.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:28:28.590 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T09:28:28.588+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123979) UID: 0 2023-11-04T09:28:28.590 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:28:28.588+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123979) UID: 0 2023-11-04T09:28:28.591 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:28:28.588+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123979) UID: 0 2023-11-04T09:28:28.591 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:28:28.588+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123979) UID: 0 2023-11-04T09:28:28.817 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:28:28.815+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640840) UID: 0 2023-11-04T09:28:28.818 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:28:28.815+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640840) UID: 0 2023-11-04T09:28:28.818 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T09:28:28.815+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640840) UID: 0 2023-11-04T09:28:28.819 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:28:28.816+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640840) UID: 0 2023-11-04T09:31:53.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:31:53.421 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:32:23.527 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:32:23.530 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:32:53.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:32:53.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:33:23.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:33:23.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:33:23.775 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:33:23.772+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640973) UID: 0 2023-11-04T09:33:23.776 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:33:23.772+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640973) UID: 0 2023-11-04T09:33:23.776 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T09:33:23.772+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640973) UID: 0 2023-11-04T09:33:23.777 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:33:23.772+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 640973) UID: 0 2023-11-04T09:36:38.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:36:38.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:37:08.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:37:08.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:37:38.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:37:38.949 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:38:09.082 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:38:09.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:38:09.231 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:38:09.229+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124240) UID: 0 2023-11-04T09:38:09.232 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T09:38:09.229+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124240) UID: 0 2023-11-04T09:38:09.233 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:38:09.232+0000 7f8a2900f700 -1 Fail to open '/proc/124240/cmdline' error = (2) No such file or directory 2023-11-04T09:38:09.233 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:38:09.232+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 124240) UID: 0 2023-11-04T09:38:09.234 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:38:09.230+0000 7fdadcd9d700 -1 Fail to open '/proc/124240/cmdline' error = (2) No such file or directory 2023-11-04T09:38:09.234 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:38:09.230+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 124240) UID: 0 2023-11-04T09:41:30.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:41:30.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:41:30.120 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:41:30.118+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124295) UID: 0 2023-11-04T09:41:30.120 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T09:41:30.118+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124295) UID: 0 2023-11-04T09:41:30.121 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:41:30.119+0000 7f8a2900f700 -1 Fail to open '/proc/124295/cmdline' error = (2) No such file or directory 2023-11-04T09:41:30.121 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:41:30.119+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 124295) UID: 0 2023-11-04T09:41:30.122 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:41:30.119+0000 7fdadcd9d700 -1 Fail to open '/proc/124295/cmdline' error = (2) No such file or directory 2023-11-04T09:41:30.122 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:41:30.119+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 124295) UID: 0 2023-11-04T09:44:54.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:44:54.471 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:44:54.534 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T09:44:54.532+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 641201) UID: 0 2023-11-04T09:44:54.535 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T09:44:54.532+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 641201) UID: 0 2023-11-04T09:48:12.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:48:12.757 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:48:43.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:48:43.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:49:13.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:49:13.130 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:49:43.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:49:43.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:50:13.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:50:13.814 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:50:44.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:50:44.207 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:50:44.267 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:50:44.266+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 641385) UID: 0 2023-11-04T09:50:44.267 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:50:44.266+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 641385) UID: 0 2023-11-04T09:50:44.268 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T09:50:44.266+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 641385) UID: 0 2023-11-04T09:50:44.268 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:50:44.266+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 641385) UID: 0 2023-11-04T09:53:09.150 INFO:teuthology.orchestra.run.smithi003.stdout:generic/521 6354s 2023-11-04T09:53:09.174 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/521 2023-11-04T09:53:09.175 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T09:53:09.177 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T09:53:09.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T09:53:09.635 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2023-11-04T09:53:10.312 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T09:53:10.317 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T09:53:10.319 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T09:53:10.326 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T09:53:10.326 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T09:54:06.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:54:06.408 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:54:36.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:54:36.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:55:06.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:55:06.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:55:36.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:55:36.884 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:56:07.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:56:07.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:56:07.056 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T09:56:07.054+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 124872) UID: 0 2023-11-04T09:56:07.056 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T09:56:07.054+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 124872) UID: 0 2023-11-04T09:56:07.057 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T09:56:07.054+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 124872) UID: 0 2023-11-04T09:56:07.057 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T09:56:07.053+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124872) UID: 0 2023-11-04T09:59:25.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:59:25.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T09:59:25.767 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T09:59:25.765+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642080) UID: 0 2023-11-04T09:59:25.767 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T09:59:25.765+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642080) UID: 0 2023-11-04T09:59:25.768 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T09:59:25.765+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642080) UID: 0 2023-11-04T09:59:25.769 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T09:59:25.765+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642080) UID: 0 2023-11-04T10:03:13.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:03:13.966 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:03:14.074 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:03:14.072+0000 7eff274c1700 -1 received signal: Hangup from (PID: 124992) UID: 0 2023-11-04T10:03:14.075 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:03:14.072+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 124992) UID: 0 2023-11-04T10:03:14.079 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:03:14.073+0000 7fdadcd9d700 -1 Fail to open '/proc/124992/cmdline' error = (2) No such file or directory 2023-11-04T10:03:14.079 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:03:14.073+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 124992) UID: 0 2023-11-04T10:03:14.079 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:03:14.073+0000 7f8a2900f700 -1 Fail to open '/proc/124992/cmdline' error = (2) No such file or directory 2023-11-04T10:03:14.079 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:03:14.073+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 124992) UID: 0 2023-11-04T10:06:43.848 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:06:43.853 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:07:14.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:07:14.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:07:44.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:07:44.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:08:14.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:08:14.463 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:08:14.528 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:08:14.526+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125123) UID: 0 2023-11-04T10:08:14.529 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:08:14.526+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125123) UID: 0 2023-11-04T10:08:14.529 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:08:14.526+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125123) UID: 0 2023-11-04T10:08:14.530 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:08:14.526+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125123) UID: 0 2023-11-04T10:11:35.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:11:35.869 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:11:35.949 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T10:11:35.946+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642333) UID: 0 2023-11-04T10:11:35.949 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:11:35.948+0000 7f7dba50c700 -1 Fail to open '/proc/642333/cmdline' error = (2) No such file or directory 2023-11-04T10:11:35.950 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:11:35.948+0000 7f7dba50c700 -1 received signal: Hangup from (PID: 642333) UID: 0 2023-11-04T10:11:35.950 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:11:35.948+0000 7f58ab082700 -1 Fail to open '/proc/642333/cmdline' error = (2) No such file or directory 2023-11-04T10:11:35.950 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:11:35.948+0000 7f58ab082700 -1 received signal: Hangup from (PID: 642333) UID: 0 2023-11-04T10:11:35.954 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:11:35.952+0000 7f97f5987700 -1 Fail to open '/proc/642333/cmdline' error = (2) No such file or directory 2023-11-04T10:11:35.954 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:11:35.952+0000 7f97f5987700 -1 received signal: Hangup from (PID: 642333) UID: 0 2023-11-04T10:15:04.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:15:04.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:15:35.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:15:35.117 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:15:35.173 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T10:15:35.171+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642410) UID: 0 2023-11-04T10:15:35.173 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:15:35.171+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642410) UID: 0 2023-11-04T10:15:35.174 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:15:35.172+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642410) UID: 0 2023-11-04T10:15:35.174 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:15:35.172+0000 7f58ab082700 -1 Fail to open '/proc/642410/cmdline' error = (2) No such file or directory 2023-11-04T10:15:35.175 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:15:35.172+0000 7f58ab082700 -1 received signal: Hangup from (PID: 642410) UID: 0 2023-11-04T10:19:06.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:19:06.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:19:36.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:19:36.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:20:07.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:20:07.287 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:20:37.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:20:37.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:21:07.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:21:07.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:21:37.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:21:37.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:22:07.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:22:07.823 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:22:37.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:22:37.987 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:23:08.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:23:08.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:23:38.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:23:38.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:24:08.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:24:08.774 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:24:38.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:24:38.875 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:25:09.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:25:09.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:25:09.085 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:25:09.083+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642766) UID: 0 2023-11-04T10:25:09.086 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:25:09.084+0000 7f58ab082700 -1 received signal: Hangup from (PID: 642766) UID: 0 2023-11-04T10:25:09.086 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T10:25:09.083+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642766) UID: 0 2023-11-04T10:25:09.088 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:25:09.083+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 642766) UID: 0 2023-11-04T10:28:34.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:28:34.240 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:28:34.327 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:28:34.326+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125662) UID: 0 2023-11-04T10:28:34.328 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:28:34.327+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 125662) UID: 0 2023-11-04T10:28:34.330 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:28:34.329+0000 7eff274c1700 -1 Fail to open '/proc/125662/cmdline' error = (2) No such file or directory 2023-11-04T10:28:34.331 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:28:34.329+0000 7eff274c1700 -1 received signal: Hangup from (PID: 125662) UID: 0 2023-11-04T10:28:34.331 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:28:34.329+0000 7f8a2900f700 -1 Fail to open '/proc/125662/cmdline' error = (2) No such file or directory 2023-11-04T10:28:34.331 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:28:34.329+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 125662) UID: 0 2023-11-04T10:31:53.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:31:53.282 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:32:23.388 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:32:23.392 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:32:53.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:32:53.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:33:23.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:33:23.684 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:33:23.762 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:33:23.760+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125794) UID: 0 2023-11-04T10:33:23.763 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:33:23.761+0000 7f8a2900f700 -1 received signal: Hangup from (PID: 125794) UID: 0 2023-11-04T10:33:23.764 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:33:23.763+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 125794) UID: 0 2023-11-04T10:33:23.769 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:33:23.760+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125794) UID: 0 2023-11-04T10:36:35.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:36:35.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:36:35.766 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:36:35.762+0000 7eff274c1700 -1 received signal: Hangup from (PID: 125843) UID: 0 2023-11-04T10:36:35.768 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:36:35.762+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125843) UID: 0 2023-11-04T10:36:35.769 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:36:35.768+0000 7f1044bc0700 -1 Fail to open '/proc/125843/cmdline' error = (2) No such file or directory 2023-11-04T10:36:35.769 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:36:35.768+0000 7f1044bc0700 -1 received signal: Hangup from (PID: 125843) UID: 0 2023-11-04T10:36:35.770 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:36:35.768+0000 7fdadcd9d700 -1 Fail to open '/proc/125843/cmdline' error = (2) No such file or directory 2023-11-04T10:36:35.770 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:36:35.768+0000 7fdadcd9d700 -1 received signal: Hangup from (PID: 125843) UID: 0 2023-11-04T10:39:28.845 INFO:teuthology.orchestra.run.smithi188.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.5.log.1: 2023-11-04T10:39:28.845 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: stdin: file size changed while zipping 2023-11-04T10:40:00.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:40:00.599 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:40:30.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:40:30.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:41:01.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:41:01.027 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:41:01.090 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:41:01.089+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643104) UID: 0 2023-11-04T10:41:01.091 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T10:41:01.089+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643104) UID: 0 2023-11-04T10:41:01.091 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:41:01.089+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643104) UID: 0 2023-11-04T10:41:01.092 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:41:01.090+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643104) UID: 0 2023-11-04T10:44:26.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:44:26.575 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:44:26.642 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T10:44:26.640+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126003) UID: 0 2023-11-04T10:44:26.642 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T10:44:26.640+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126003) UID: 0 2023-11-04T10:44:26.643 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T10:44:26.640+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126003) UID: 0 2023-11-04T10:44:26.643 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T10:44:26.640+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126003) UID: 0 2023-11-04T10:47:53.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:47:53.463 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:47:53.538 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:47:53.536+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643208) UID: 0 2023-11-04T10:47:53.539 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T10:47:53.536+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643208) UID: 0 2023-11-04T10:47:53.540 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:47:53.536+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643208) UID: 0 2023-11-04T10:47:53.540 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:47:53.537+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643208) UID: 0 2023-11-04T10:51:18.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:51:18.678 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:51:18.737 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T10:51:18.736+0000 7fda51433700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 643270) UID: 0 2023-11-04T10:51:18.737 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T10:51:18.736+0000 7f0aafeef700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 643270) UID: 0 2023-11-04T10:54:06.965 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T10:54:06.963+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643290) UID: 0 2023-11-04T10:54:06.966 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T10:54:06.964+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643290) UID: 0 2023-11-04T10:54:06.968 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T10:54:06.963+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 643290) UID: 0 2023-11-04T10:54:06.969 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:54:06.964+0000 7f97f5987700 -1 Fail to open '/proc/643290/cmdline' error = (2) No such file or directory 2023-11-04T10:54:06.969 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T10:54:06.964+0000 7f97f5987700 -1 received signal: Hangup from (PID: 643290) UID: 0 2023-11-04T10:55:29.571 INFO:teuthology.orchestra.run.smithi003.stdout:generic/522 3738s 2023-11-04T10:55:29.719 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/522 2023-11-04T10:55:29.720 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:29.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:29.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:29.793 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2023-11-04T10:55:33.366 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:33.371 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:33.371 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:33.372 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:33.373 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:34.542 INFO:teuthology.orchestra.run.smithi003.stdout:generic/523 1s 2023-11-04T10:55:34.559 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/523 2023-11-04T10:55:34.559 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:34.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:34.576 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:34.577 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2023-11-04T10:55:35.066 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:35.071 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:35.074 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:35.076 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:35.076 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:41.575 INFO:teuthology.orchestra.run.smithi003.stdout:generic/524 6s 2023-11-04T10:55:41.590 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/524 2023-11-04T10:55:41.590 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:41.595 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:41.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:41.606 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2023-11-04T10:55:42.072 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:42.086 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:42.087 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:42.088 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:43.025 INFO:teuthology.orchestra.run.smithi003.stdout:generic/525 [not run] filesystem does not support huge file size 2023-11-04T10:55:43.053 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/525 2023-11-04T10:55:43.053 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/525 2023-11-04T10:55:43.053 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:43.055 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:43.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:43.089 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2023-11-04T10:55:43.542 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:43.547 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:43.550 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:43.550 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:43.550 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:44.343 INFO:teuthology.orchestra.run.smithi003.stdout:generic/526 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T10:55:44.354 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/526 2023-11-04T10:55:44.354 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/526 2023-11-04T10:55:44.354 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:44.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:44.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:44.403 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2023-11-04T10:55:44.892 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:44.897 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:44.898 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:44.899 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:44.899 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:generic/527 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T10:55:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/527 2023-11-04T10:55:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/527 2023-11-04T10:55:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:45.548 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:45.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:45.592 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2023-11-04T10:55:45.986 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:46.002 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:46.003 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:46.004 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:46.004 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:46.751 INFO:teuthology.orchestra.run.smithi003.stdout:generic/528 0s 2023-11-04T10:55:46.765 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/528 2023-11-04T10:55:46.765 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:46.768 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:47.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:47.200 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2023-11-04T10:55:47.710 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:47.717 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:47.718 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:47.719 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:47.719 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:48.857 INFO:teuthology.orchestra.run.smithi003.stdout:generic/529 0s 2023-11-04T10:55:48.873 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/529 2023-11-04T10:55:48.874 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:48.876 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:48.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:48.890 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2023-11-04T10:55:49.384 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:49.389 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:49.389 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:49.391 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:49.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:50.214 INFO:teuthology.orchestra.run.smithi003.stdout:generic/530 [not run] ceph does not support shutdown 2023-11-04T10:55:50.258 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/530 2023-11-04T10:55:50.258 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/530 2023-11-04T10:55:50.259 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:50.262 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:50.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:50.306 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2023-11-04T10:55:50.825 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:50.831 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:50.839 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:50.840 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:50.840 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:51.410 INFO:teuthology.orchestra.run.smithi003.stdout:generic/531 [not run] O_TMPFILE is not supported 2023-11-04T10:55:51.421 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/531 2023-11-04T10:55:51.421 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/531 2023-11-04T10:55:51.421 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:51.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:51.467 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2023-11-04T10:55:52.017 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:52.032 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:52.033 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:52.034 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:52.747 INFO:teuthology.orchestra.run.smithi003.stdout:generic/532 0s 2023-11-04T10:55:52.762 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/532 2023-11-04T10:55:52.762 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:52.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:52.822 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2023-11-04T10:55:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:53.309 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:53.311 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:53.315 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:53.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:54.912 INFO:teuthology.orchestra.run.smithi003.stdout:generic/533 1s 2023-11-04T10:55:54.927 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/533 2023-11-04T10:55:54.927 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:54.930 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:54.982 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:54.983 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2023-11-04T10:55:55.484 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:55.494 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:55.494 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:55.495 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:55.495 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:56.117 INFO:teuthology.orchestra.run.smithi003.stdout:generic/534 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T10:55:56.126 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/534 2023-11-04T10:55:56.127 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/534 2023-11-04T10:55:56.127 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:56.130 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:56.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:56.173 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2023-11-04T10:55:56.661 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:56.665 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:56.666 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:56.668 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:56.668 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:57.336 INFO:teuthology.orchestra.run.smithi003.stdout:generic/535 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T10:55:57.340 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/535 2023-11-04T10:55:57.340 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/535 2023-11-04T10:55:57.340 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:57.343 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:57.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:57.372 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2023-11-04T10:55:57.825 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:57.833 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:57.834 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:57.841 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:58.694 INFO:teuthology.orchestra.run.smithi003.stdout:generic/536 [not run] ceph does not support shutdown 2023-11-04T10:55:58.731 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/536 2023-11-04T10:55:58.731 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/536 2023-11-04T10:55:58.731 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:55:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:55:58.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:55:58.777 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2023-11-04T10:55:59.244 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:55:59.255 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:55:59.256 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:55:59.257 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:55:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:56:00.060 INFO:teuthology.orchestra.run.smithi003.stdout:generic/537 [not run] FSTRIM not supported 2023-11-04T10:56:00.094 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/537 2023-11-04T10:56:00.094 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/537 2023-11-04T10:56:00.095 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T10:56:00.108 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:56:00.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T10:56:00.152 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2023-11-04T10:56:00.543 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T10:56:00.550 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T10:56:00.551 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T10:56:00.553 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T10:56:00.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T10:57:52.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:57:52.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:58:22.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:58:22.863 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:58:52.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:58:52.961 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:59:23.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:59:23.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:59:53.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T10:59:53.196 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:00:23.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:00:23.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:00:23.795 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T11:00:23.795+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 655192) UID: 0 2023-11-04T11:00:23.796 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T11:00:23.795+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 655192) UID: 0 2023-11-04T11:00:23.796 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T11:00:23.795+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 655192) UID: 0 2023-11-04T11:00:23.798 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T11:00:23.794+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 655192) UID: 0 2023-11-04T11:03:53.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:03:53.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:04:23.531 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:04:23.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:04:53.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:04:53.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:05:22.101 INFO:teuthology.orchestra.run.smithi003.stdout:generic/538 562s 2023-11-04T11:05:22.107 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/538 2023-11-04T11:05:22.107 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:22.109 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:22.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:22.463 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2023-11-04T11:05:23.019 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:23.024 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:23.026 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:23.028 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:23.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:24.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:05:24.216 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:05:24.369 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:05:24.367+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126964) UID: 0 2023-11-04T11:05:24.369 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:05:24.367+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126964) UID: 0 2023-11-04T11:05:24.370 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:05:24.368+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126964) UID: 0 2023-11-04T11:05:24.370 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:05:24.368+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126964) UID: 0 2023-11-04T11:05:25.739 INFO:teuthology.orchestra.run.smithi003.stdout:generic/539 2s 2023-11-04T11:05:25.746 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/539 2023-11-04T11:05:25.746 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:25.755 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:26.165 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:26.166 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2023-11-04T11:05:26.520 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:26.529 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:26.530 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:26.531 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:26.531 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:27.693 INFO:teuthology.orchestra.run.smithi003.stdout:generic/540 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T11:05:27.797 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/540 2023-11-04T11:05:27.797 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/540 2023-11-04T11:05:27.797 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:27.803 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:27.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:27.906 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2023-11-04T11:05:28.277 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:28.282 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:28.282 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:28.283 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:28.283 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:29.516 INFO:teuthology.orchestra.run.smithi003.stdout:generic/541 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T11:05:29.545 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/541 2023-11-04T11:05:29.545 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/541 2023-11-04T11:05:29.546 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:29.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:29.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:29.631 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2023-11-04T11:05:29.941 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:29.946 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:29.947 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:29.951 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:29.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:31.124 INFO:teuthology.orchestra.run.smithi003.stdout:generic/542 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T11:05:31.130 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/542 2023-11-04T11:05:31.131 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/542 2023-11-04T11:05:31.131 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:31.133 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:31.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:31.449 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2023-11-04T11:05:31.815 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:31.819 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:31.820 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:31.821 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:31.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:32.981 INFO:teuthology.orchestra.run.smithi003.stdout:generic/543 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T11:05:32.988 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/543 2023-11-04T11:05:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/543 2023-11-04T11:05:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:33.027 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:33.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:33.107 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2023-11-04T11:05:33.447 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:33.454 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:33.455 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:33.456 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:33.456 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:34.582 INFO:teuthology.orchestra.run.smithi003.stdout:generic/544 [not run] Reflink not supported by scratch filesystem type: ceph 2023-11-04T11:05:34.670 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/544 2023-11-04T11:05:34.670 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/544 2023-11-04T11:05:34.670 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:34.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:34.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:34.746 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2023-11-04T11:05:35.069 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:35.073 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:35.074 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:35.076 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:35.076 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:35.720 INFO:teuthology.orchestra.run.smithi003.stdout:generic/545 [not run] file system doesn't support chattr +i 2023-11-04T11:05:35.729 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/545 2023-11-04T11:05:35.730 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/545 2023-11-04T11:05:35.730 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:35.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:35.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:35.788 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2023-11-04T11:05:36.164 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:36.169 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:36.170 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:36.171 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:36.171 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:36.729 INFO:teuthology.orchestra.run.smithi003.stdout:generic/546 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-11-04T11:05:36.734 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/546 2023-11-04T11:05:36.734 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/546 2023-11-04T11:05:36.734 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:36.737 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:36.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:36.783 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2023-11-04T11:05:37.127 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:37.131 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:37.131 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:37.133 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:37.133 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:37.667 INFO:teuthology.orchestra.run.smithi003.stdout:generic/547 [not run] require 172.21.15.3:6789:/scratch to be valid block disk 2023-11-04T11:05:37.675 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/547 2023-11-04T11:05:37.676 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/547 2023-11-04T11:05:37.676 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:37.678 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:37.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:37.716 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2023-11-04T11:05:38.092 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:38.106 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:38.106 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:38.107 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:38.107 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:38.983 INFO:teuthology.orchestra.run.smithi003.stdout:generic/548 [not run] kernel does not support ceph encryption 2023-11-04T11:05:39.007 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/548 2023-11-04T11:05:39.007 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/548 2023-11-04T11:05:39.007 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:39.010 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:39.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:39.038 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2023-11-04T11:05:39.411 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:39.430 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:39.432 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:39.434 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:39.434 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:40.285 INFO:teuthology.orchestra.run.smithi003.stdout:generic/549 [not run] kernel does not support ceph encryption 2023-11-04T11:05:40.324 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/549 2023-11-04T11:05:40.325 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/549 2023-11-04T11:05:40.325 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:40.326 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:40.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:40.363 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2023-11-04T11:05:40.707 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:40.713 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:40.713 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:40.715 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:40.715 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:41.599 INFO:teuthology.orchestra.run.smithi003.stdout:generic/550 [not run] kernel does not support ceph encryption 2023-11-04T11:05:41.626 INFO:teuthology.orchestra.run.smithi003.stdout:Ran: generic/550 2023-11-04T11:05:41.629 INFO:teuthology.orchestra.run.smithi003.stdout:Not run: generic/550 2023-11-04T11:05:41.629 INFO:teuthology.orchestra.run.smithi003.stdout:Passed all 1 tests 2023-11-04T11:05:41.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:05:41.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-11-04T11:05:41.667 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/tmp.ByRml5cCmzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2023-11-04T11:05:41.993 INFO:teuthology.orchestra.run.smithi003.stdout:FSTYP -- ceph 2023-11-04T11:05:41.998 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM -- Linux/x86_64 smithi003 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-11-04T11:05:41.999 INFO:teuthology.orchestra.run.smithi003.stdout:MKFS_OPTIONS -- 172.21.15.3:6789:/scratch 2023-11-04T11:05:42.000 INFO:teuthology.orchestra.run.smithi003.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQA71kVl0t9JHhAA7TxXeMjkoNV0y8y9qWtgtg== -o context=system_u:object_r:root_t:s0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch 2023-11-04T11:05:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:08:48.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:08:48.950 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:08:49.058 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:08:49.057+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128280) UID: 0 2023-11-04T11:08:49.059 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:08:49.058+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128280) UID: 0 2023-11-04T11:08:49.061 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:08:49.059+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128280) UID: 0 2023-11-04T11:08:49.062 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:08:49.059+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128280) UID: 0 2023-11-04T11:12:23.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:12:23.977 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:12:24.068 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:12:24.067+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128560) UID: 0 2023-11-04T11:12:24.070 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:12:24.068+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128560) UID: 0 2023-11-04T11:12:24.070 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:12:24.069+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128560) UID: 0 2023-11-04T11:12:24.071 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:12:24.069+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 128560) UID: 0 2023-11-04T11:16:04.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:16:04.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:16:04.232 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T11:16:04.230+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 667541) UID: 0 2023-11-04T11:16:04.234 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T11:16:04.233+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 667541) UID: 0 2023-11-04T11:16:04.236 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T11:16:04.230+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 667541) UID: 0 2023-11-04T11:16:04.237 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T11:16:04.234+0000 7f1f6ddbb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 667541) UID: 0 2023-11-04T11:18:15.316 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T11:18:15.315+0000 7f1f64197700 -1 bdev(0x557c50496700 /var/lib/ceph/osd/ceph-2/block) _aio_thread got r=-28 ((28) No space left on device) 2023-11-04T11:18:15.403 INFO:tasks.ceph.osd.2.smithi003.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f1f64197700 time 2023-11-04T11:18:15.380314+0000 2023-11-04T11:18:15.404 INFO:tasks.ceph.osd.2.smithi003.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-11-04T11:18:15.404 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.404 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x557c4c41edc1] 2023-11-04T11:18:15.404 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.405 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.405 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.405 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: clone() 2023-11-04T11:18:15.405 INFO:tasks.ceph.osd.2.smithi003.stderr:*** Caught signal (Aborted) ** 2023-11-04T11:18:15.405 INFO:tasks.ceph.osd.2.smithi003.stderr: in thread 7f1f64197700 thread_name:bstore_aio 2023-11-04T11:18:15.406 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T11:18:15.403+0000 7f1f64197700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f1f64197700 time 2023-11-04T11:18:15.380314+0000 2023-11-04T11:18:15.406 INFO:tasks.ceph.osd.2.smithi003.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-11-04T11:18:15.406 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.406 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.407 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x557c4c41edc1] 2023-11-04T11:18:15.407 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.407 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.407 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.407 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: clone() 2023-11-04T11:18:15.408 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.408 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.408 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f1f764c2ce0] 2023-11-04T11:18:15.409 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: gsignal() 2023-11-04T11:18:15.409 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: abort() 2023-11-04T11:18:15.409 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x557c4c41ee81] 2023-11-04T11:18:15.409 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.410 INFO:tasks.ceph.osd.2.smithi003.stderr: 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.410 INFO:tasks.ceph.osd.2.smithi003.stderr: 7: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.410 INFO:tasks.ceph.osd.2.smithi003.stderr: 8: clone() 2023-11-04T11:18:15.410 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-11-04T11:18:15.405+0000 7f1f64197700 -1 *** Caught signal (Aborted) ** 2023-11-04T11:18:15.410 INFO:tasks.ceph.osd.2.smithi003.stderr: in thread 7f1f64197700 thread_name:bstore_aio 2023-11-04T11:18:15.411 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.411 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.411 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f1f764c2ce0] 2023-11-04T11:18:15.411 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: gsignal() 2023-11-04T11:18:15.411 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: abort() 2023-11-04T11:18:15.412 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x557c4c41ee81] 2023-11-04T11:18:15.412 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.412 INFO:tasks.ceph.osd.2.smithi003.stderr: 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.412 INFO:tasks.ceph.osd.2.smithi003.stderr: 7: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.412 INFO:tasks.ceph.osd.2.smithi003.stderr: 8: clone() 2023-11-04T11:18:15.413 INFO:tasks.ceph.osd.2.smithi003.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-04T11:18:15.413 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.422 INFO:tasks.ceph.osd.2.smithi003.stderr: -6> 2023-11-04T11:18:15.315+0000 7f1f64197700 -1 bdev(0x557c50496700 /var/lib/ceph/osd/ceph-2/block) _aio_thread got r=-28 ((28) No space left on device) 2023-11-04T11:18:15.423 INFO:tasks.ceph.osd.2.smithi003.stderr: -1> 2023-11-04T11:18:15.403+0000 7f1f64197700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f1f64197700 time 2023-11-04T11:18:15.380314+0000 2023-11-04T11:18:15.423 INFO:tasks.ceph.osd.2.smithi003.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-11-04T11:18:15.423 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.423 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.424 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x557c4c41edc1] 2023-11-04T11:18:15.424 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.424 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.424 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.424 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: clone() 2023-11-04T11:18:15.424 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.425 INFO:tasks.ceph.osd.2.smithi003.stderr: 0> 2023-11-04T11:18:15.405+0000 7f1f64197700 -1 *** Caught signal (Aborted) ** 2023-11-04T11:18:15.425 INFO:tasks.ceph.osd.2.smithi003.stderr: in thread 7f1f64197700 thread_name:bstore_aio 2023-11-04T11:18:15.425 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.425 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.425 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f1f764c2ce0] 2023-11-04T11:18:15.426 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: gsignal() 2023-11-04T11:18:15.426 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: abort() 2023-11-04T11:18:15.426 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x557c4c41ee81] 2023-11-04T11:18:15.426 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.426 INFO:tasks.ceph.osd.2.smithi003.stderr: 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.427 INFO:tasks.ceph.osd.2.smithi003.stderr: 7: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.427 INFO:tasks.ceph.osd.2.smithi003.stderr: 8: clone() 2023-11-04T11:18:15.427 INFO:tasks.ceph.osd.2.smithi003.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-04T11:18:15.427 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.437 INFO:tasks.ceph.osd.2.smithi003.stderr: -130> 2023-11-04T11:18:15.315+0000 7f1f64197700 -1 bdev(0x557c50496700 /var/lib/ceph/osd/ceph-2/block) _aio_thread got r=-28 ((28) No space left on device) 2023-11-04T11:18:15.437 INFO:tasks.ceph.osd.2.smithi003.stderr: -125> 2023-11-04T11:18:15.403+0000 7f1f64197700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f1f64197700 time 2023-11-04T11:18:15.380314+0000 2023-11-04T11:18:15.437 INFO:tasks.ceph.osd.2.smithi003.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7050-g826ae420/rpm/el8/BUILD/ceph-18.0.0-7050-g826ae420/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-11-04T11:18:15.437 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.438 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.438 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x557c4c41edc1] 2023-11-04T11:18:15.438 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.438 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.438 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.439 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: clone() 2023-11-04T11:18:15.439 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.439 INFO:tasks.ceph.osd.2.smithi003.stderr: -124> 2023-11-04T11:18:15.405+0000 7f1f64197700 -1 *** Caught signal (Aborted) ** 2023-11-04T11:18:15.439 INFO:tasks.ceph.osd.2.smithi003.stderr: in thread 7f1f64197700 thread_name:bstore_aio 2023-11-04T11:18:15.439 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.439 INFO:tasks.ceph.osd.2.smithi003.stderr: ceph version 18.0.0-7050-g826ae420 (826ae4205210243850fc3f7bc3f8aea0d1f5ced2) reef (dev) 2023-11-04T11:18:15.440 INFO:tasks.ceph.osd.2.smithi003.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f1f764c2ce0] 2023-11-04T11:18:15.440 INFO:tasks.ceph.osd.2.smithi003.stderr: 2: gsignal() 2023-11-04T11:18:15.440 INFO:tasks.ceph.osd.2.smithi003.stderr: 3: abort() 2023-11-04T11:18:15.440 INFO:tasks.ceph.osd.2.smithi003.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x557c4c41ee81] 2023-11-04T11:18:15.440 INFO:tasks.ceph.osd.2.smithi003.stderr: 5: (KernelDevice::_aio_thread()+0x18a6) [0x557c4d095f96] 2023-11-04T11:18:15.441 INFO:tasks.ceph.osd.2.smithi003.stderr: 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x557c4d09df91] 2023-11-04T11:18:15.441 INFO:tasks.ceph.osd.2.smithi003.stderr: 7: /lib64/libpthread.so.0(+0x81cf) [0x7f1f764b81cf] 2023-11-04T11:18:15.441 INFO:tasks.ceph.osd.2.smithi003.stderr: 8: clone() 2023-11-04T11:18:15.441 INFO:tasks.ceph.osd.2.smithi003.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-04T11:18:15.441 INFO:tasks.ceph.osd.2.smithi003.stderr: 2023-11-04T11:18:15.748 INFO:tasks.ceph.osd.2.smithi003.stderr:daemon-helper: command failed with exit status 5 2023-11-04T11:18:17.641 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-04T11:18:24.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~7s 2023-11-04T11:18:30.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~13s 2023-11-04T11:18:37.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2023-11-04T11:18:44.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~26s 2023-11-04T11:18:50.716 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~33s 2023-11-04T11:18:57.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~40s 2023-11-04T11:19:03.940 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~46s 2023-11-04T11:19:10.552 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~53s 2023-11-04T11:19:17.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~60s 2023-11-04T11:19:23.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~66s 2023-11-04T11:19:30.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~73s 2023-11-04T11:19:31.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:19:31.164 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:19:31.307 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:19:31.306+0000 7eff274c1700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 129521) UID: 0 2023-11-04T11:19:31.308 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:19:31.306+0000 7f1044bc0700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 129521) UID: 0 2023-11-04T11:19:31.308 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:19:31.306+0000 7f8a2900f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 129521) UID: 0 2023-11-04T11:19:31.309 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:19:31.307+0000 7fdadcd9d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 129521) UID: 0 2023-11-04T11:19:36.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~79s 2023-11-04T11:19:43.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~86s 2023-11-04T11:19:50.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~93s 2023-11-04T11:19:56.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~99s 2023-11-04T11:20:03.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~106s 2023-11-04T11:20:10.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~112s 2023-11-04T11:20:16.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~119s 2023-11-04T11:20:23.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~126s 2023-11-04T11:20:29.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~132s 2023-11-04T11:20:36.506 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~139s 2023-11-04T11:20:43.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~145s 2023-11-04T11:20:49.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~152s 2023-11-04T11:20:56.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~159s 2023-11-04T11:21:02.955 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2023-11-04T11:21:09.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~172s 2023-11-04T11:21:16.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~179s 2023-11-04T11:21:22.793 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~185s 2023-11-04T11:21:29.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~192s 2023-11-04T11:21:36.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~198s 2023-11-04T11:21:42.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~205s 2023-11-04T11:21:49.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~212s 2023-11-04T11:21:55.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~218s 2023-11-04T11:22:02.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~225s 2023-11-04T11:22:09.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~231s 2023-11-04T11:22:15.699 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~238s 2023-11-04T11:22:22.312 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2023-11-04T11:22:28.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~251s 2023-11-04T11:22:35.535 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~258s 2023-11-04T11:22:42.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~265s 2023-11-04T11:22:48.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~271s 2023-11-04T11:22:55.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~278s 2023-11-04T11:23:01.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~284s 2023-11-04T11:23:03.019 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:23:03.022 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:23:03.081 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T11:23:03.079+0000 7f7dba50c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 668164) UID: 0 2023-11-04T11:23:03.082 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T11:23:03.079+0000 7f58ab082700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 668164) UID: 0 2023-11-04T11:23:03.082 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T11:23:03.079+0000 7f97f5987700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 668164) UID: 0 2023-11-04T11:23:08.589 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~291s 2023-11-04T11:23:15.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~298s 2023-11-04T11:23:21.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~304s 2023-11-04T11:23:21.815 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-11-04T11:23:21.815 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T11:23:21.854 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T11:23:21.919 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T11:23:21.919 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T11:23:21.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-04T11:23:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.825+0000 7f1b963ae700 1 Processor -- start 2023-11-04T11:23:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.825+0000 7f1b963ae700 1 -- start start 2023-11-04T11:23:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.825+0000 7f1b963ae700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b9015dc50 0x7f1b900716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.825+0000 7f1b963ae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b9006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b963ae700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b900720a0 con 0x7f1b9006dcd0 2023-11-04T11:23:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b963ae700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9015e030 con 0x7f1b9015dc50 2023-11-04T11:23:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b963ae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9006e140 con 0x7f1b90071c00 2023-11-04T11:23:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b9494b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1b9006dcd0 0x7f1b9006ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42112/0 (socket says 172.21.15.3:42112) 2023-11-04T11:23:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b9494b700 1 -- 172.21.15.3:0/164442553 learned_addr learned my addr 172.21.15.3:0/164442553 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T11:23:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8f7fe700 1 --2- 172.21.15.3:0/164442553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b9006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8ffff700 1 --2- 172.21.15.3:0/164442553 >> v2:172.21.15.188:3300/0 conn(0x7f1b9015dc50 0x7f1b900716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8effd700 1 -- 172.21.15.3:0/164442553 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911439630 0 0) 0x7f1b900720a0 con 0x7f1b9006dcd0 2023-11-04T11:23:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8effd700 1 -- 172.21.15.3:0/164442553 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b780035b0 con 0x7f1b9006dcd0 2023-11-04T11:23:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8f7fe700 1 -- 172.21.15.3:0/164442553 >> v2:172.21.15.188:3300/0 conn(0x7f1b9015dc50 msgr2=0x7f1b900716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T11:23:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8f7fe700 1 --2- 172.21.15.3:0/164442553 >> v2:172.21.15.188:3300/0 conn(0x7f1b9015dc50 0x7f1b900716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8f7fe700 1 -- 172.21.15.3:0/164442553 >> v1:172.21.15.3:6789/0 conn(0x7f1b9006dcd0 legacy=0x7f1b9006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8f7fe700 1 -- 172.21.15.3:0/164442553 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90072380 con 0x7f1b90071c00 2023-11-04T11:23:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.826+0000 7f1b8ffff700 1 --2- 172.21.15.3:0/164442553 >> v2:172.21.15.188:3300/0 conn(0x7f1b9015dc50 0x7f1b900716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T11:23:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.827+0000 7f1b8f7fe700 1 --2- 172.21.15.3:0/164442553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b9006d790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009820 tx=0x7f1b84009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afee477e497927b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.827+0000 7f1b8effd700 1 -- 172.21.15.3:0/164442553 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84005e30 con 0x7f1b90071c00 2023-11-04T11:23:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.827+0000 7f1b8effd700 1 -- 172.21.15.3:0/164442553 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b84006030 con 0x7f1b90071c00 2023-11-04T11:23:24.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.827+0000 7f1b8effd700 1 -- 172.21.15.3:0/164442553 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840063c0 con 0x7f1b90071c00 2023-11-04T11:23:24.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.827+0000 7f1b963ae700 1 -- 172.21.15.3:0/164442553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 msgr2=0x7f1b9006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:24.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.827+0000 7f1b963ae700 1 --2- 172.21.15.3:0/164442553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b9006d790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009820 tx=0x7f1b84009b30 comp rx=0 tx=0).stop 2023-11-04T11:23:24.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.828+0000 7f1b963ae700 1 -- 172.21.15.3:0/164442553 shutdown_connections 2023-11-04T11:23:24.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.828+0000 7f1b963ae700 1 --2- 172.21.15.3:0/164442553 >> v2:172.21.15.188:3300/0 conn(0x7f1b9015dc50 0x7f1b900716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.828+0000 7f1b963ae700 1 --2- 172.21.15.3:0/164442553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b9006d790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.828+0000 7f1b963ae700 1 -- 172.21.15.3:0/164442553 >> 172.21.15.3:0/164442553 conn(0x7f1b900cfd50 msgr2=0x7f1b900d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.828+0000 7f1b963ae700 1 -- 172.21.15.3:0/164442553 shutdown_connections 2023-11-04T11:23:24.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.828+0000 7f1b963ae700 1 -- 172.21.15.3:0/164442553 wait complete. 2023-11-04T11:23:24.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.829+0000 7f1b963ae700 1 Processor -- start 2023-11-04T11:23:24.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.829+0000 7f1b963ae700 1 -- start start 2023-11-04T11:23:24.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b963ae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9006dcd0 0x7f1b900c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:24.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b963ae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b900ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b963ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 0x7f1b900ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b963ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c4e10 con 0x7f1b9015dc50 2023-11-04T11:23:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b963ae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c4190 con 0x7f1b9006dcd0 2023-11-04T11:23:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b963ae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c4470 con 0x7f1b90071c00 2023-11-04T11:23:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 0x7f1b900ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 0x7f1b900ad5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36770/0 (socket says 172.21.15.3:36770) 2023-11-04T11:23:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 -- 172.21.15.3:0/3668990619 learned_addr learned my addr 172.21.15.3:0/3668990619 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b8ffff700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9006dcd0 0x7f1b900c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b8f7fe700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b900ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 -- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 msgr2=0x7f1b900ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b900ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 -- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9006dcd0 msgr2=0x7f1b900c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9006dcd0 0x7f1b900c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.830+0000 7f1b9494b700 1 -- 172.21.15.3:0/3668990619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094a0 con 0x7f1b9015dc50 2023-11-04T11:23:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.831+0000 7f1b9494b700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 0x7f1b900ad5c0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800cb40 tx=0x7f1b8800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.831+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800a9e0 con 0x7f1b9015dc50 2023-11-04T11:23:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.831+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8800ab40 con 0x7f1b9015dc50 2023-11-04T11:23:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.831+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88012400 con 0x7f1b9015dc50 2023-11-04T11:23:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.831+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b84009430 con 0x7f1b9015dc50 2023-11-04T11:23:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.831+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b900c4a30 con 0x7f1b9015dc50 2023-11-04T11:23:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.832+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b900c4f50 con 0x7f1b9015dc50 2023-11-04T11:23:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.833+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b8801b3b0 con 0x7f1b9015dc50 2023-11-04T11:23:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.833+0000 7f1b8cff9700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1b7c06af00 0x7f1b7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.833+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1b88089310 con 0x7f1b9015dc50 2023-11-04T11:23:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.834+0000 7f1b8ffff700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1b7c06af00 0x7f1b7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.834+0000 7f1b8ffff700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1b7c06af00 0x7f1b7c06d3c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009cd0 tx=0x7f1b80009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:24.840+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f1b900c4f50 con 0x7f1b9015dc50 2023-11-04T11:23:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.049+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b900c4f50 con 0x7f1b9015dc50 2023-11-04T11:23:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.051+0000 7f1b8cff9700 1 -- 172.21.15.3:0/3668990619 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16850 (secure 0 0 0) 0x7f1b900c4f50 con 0x7f1b9015dc50 2023-11-04T11:23:25.052 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-11-04T11:23:25.052 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":44,"fsid":"7a64e299-05b0-4fd0-819f-2262a06846bb","created":"2023-11-04T05:27:58.045456+0000","modified":"2023-11-04T11:18:16.707711+0000","last_up_change":"2023-11-04T11:18:15.679408+0000","last_in_change":"2023-11-04T05:28:00.589881+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-04T05:28:09.808516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-04T05:29:40.675722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-04T05:29:41.624805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":0.87999999523162842,"raw_score_acting":1.2000000476837158,"raw_score_stable":1.2000000476837158,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"384c5e00-e453-4d87-a383-a977aefa4762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6818","nonce":2508882777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6822","nonce":2508882777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6830","nonce":2508882777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2508882777},{"type":"v1","addr":"172.21.15.3:6826","nonce":2508882777}]},"public_addr":"172.21.15.3:6818/2508882777","cluster_addr":"172.21.15.3:6822/2508882777","heartbeat_back_addr":"172.21.15.3:6830/2508882777","heartbeat_front_addr":"172.21.15.3:6826/2508882777","state":["exists","up"]},{"osd":1,"uuid":"ed1ccd37-d112-48a3-97a6-0d1be873a282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6804","nonce":2297993387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6808","nonce":2297993387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6815","nonce":2297993387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2297993387},{"type":"v1","addr":"172.21.15.3:6812","nonce":2297993387}]},"public_addr":"172.21.15.3:6804/2297993387","cluster_addr":"172.21.15.3:6808/2297993387","heartbeat_back_addr":"172.21.15.3:6815/2297993387","heartbeat_front_addr":"172.21.15.3:6812/2297993387","state":["exists","up"]},{"osd":2,"uuid":"a31bc146-fd57-4ad8-9f9b-3a4736a7b582","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6801","nonce":3907251966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6805","nonce":3907251966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6811","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6813","nonce":3907251966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":3907251966},{"type":"v1","addr":"172.21.15.3:6809","nonce":3907251966}]},"public_addr":"172.21.15.3:6801/3907251966","cluster_addr":"172.21.15.3:6805/3907251966","heartbeat_back_addr":"172.21.15.3:6813/3907251966","heartbeat_front_addr":"172.21.15.3:6809/3907251966","state":["exists"]},{"osd":3,"uuid":"f8a58fb6-14ea-4604-9d75-0d1b237cad96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6817","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6819","nonce":4243967239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6821","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6823","nonce":4243967239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6829","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6831","nonce":4243967239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6825","nonce":4243967239},{"type":"v1","addr":"172.21.15.3:6827","nonce":4243967239}]},"public_addr":"172.21.15.3:6819/4243967239","cluster_addr":"172.21.15.3:6823/4243967239","heartbeat_back_addr":"172.21.15.3:6831/4243967239","heartbeat_front_addr":"172.21.15.3:6827/4243967239","state":["exists","up"]},{"osd":4,"uuid":"f8eeca37-9e36-4201-8c72-e9133f3b85a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6801","nonce":989266714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6803","nonce":989266714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6807","nonce":989266714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":989266714},{"type":"v1","addr":"172.21.15.188:6805","nonce":989266714}]},"public_addr":"172.21.15.188:6801/989266714","cluster_addr":"172.21.15.188:6803/989266714","heartbeat_back_addr":"172.21.15.188:6807/989266714","heartbeat_front_addr":"172.21.15.188:6805/989266714","state":["exists","up"]},{"osd":5,"uuid":"b9a3edd4-a3fc-48dc-a90f-38b5421bc2a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6816","nonce":3601495681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6819","nonce":3601495681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6823","nonce":3601495681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3601495681},{"type":"v1","addr":"172.21.15.188:6821","nonce":3601495681}]},"public_addr":"172.21.15.188:6816/3601495681","cluster_addr":"172.21.15.188:6819/3601495681","heartbeat_back_addr":"172.21.15.188:6823/3601495681","heartbeat_front_addr":"172.21.15.188:6821/3601495681","state":["exists","up"]},{"osd":6,"uuid":"f23194fe-1570-4fa1-af76-fe10e798e7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6809","nonce":830502099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6811","nonce":830502099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6815","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6817","nonce":830502099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":830502099},{"type":"v1","addr":"172.21.15.188:6813","nonce":830502099}]},"public_addr":"172.21.15.188:6809/830502099","cluster_addr":"172.21.15.188:6811/830502099","heartbeat_back_addr":"172.21.15.188:6817/830502099","heartbeat_front_addr":"172.21.15.188:6813/830502099","state":["exists","up"]},{"osd":7,"uuid":"d81eabe1-34b1-43d7-b5c5-9ac505310564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6825","nonce":1844078319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6827","nonce":1844078319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6831","nonce":1844078319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1844078319},{"type":"v1","addr":"172.21.15.188:6829","nonce":1844078319}]},"public_addr":"172.21.15.188:6825/1844078319","cluster_addr":"172.21.15.188:6827/1844078319","heartbeat_back_addr":"172.21.15.188:6831/1844078319","heartbeat_front_addr":"172.21.15.188:6829/1844078319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.797778+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.798420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.771079+0000","dead_epoch":43},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.769301+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.608095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.815178+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.786135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-04T05:28:05.870063+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-04T11:23:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1b7c06af00 msgr2=0x7f1b7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1b7c06af00 0x7f1b7c06d3c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009cd0 tx=0x7f1b80009210 comp rx=0 tx=0).stop 2023-11-04T11:23:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 msgr2=0x7f1b900ad5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 0x7f1b900ad5c0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800cb40 tx=0x7f1b8800ce50 comp rx=0 tx=0).stop 2023-11-04T11:23:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 shutdown_connections 2023-11-04T11:23:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f1b7c06af00 0x7f1b7c06d3c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9006dcd0 0x7f1b900c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b9015dc50 0x7f1b900ad5c0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 --2- 172.21.15.3:0/3668990619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b90071c00 0x7f1b900ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 >> 172.21.15.3:0/3668990619 conn(0x7f1b900cfd50 msgr2=0x7f1b900cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:25.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 shutdown_connections 2023-11-04T11:23:25.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.055+0000 7f1b963ae700 1 -- 172.21.15.3:0/3668990619 wait complete. 2023-11-04T11:23:25.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-04T05:28:09.808516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-04T05:29:40.675722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-04T05:29:41.624805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.2000000476837158, 'raw_score_stable': 1.2000000476837158, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-04T11:23:25.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-04T11:23:25.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.217+0000 7ff853f9c700 1 Processor -- start 2023-11-04T11:23:25.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.217+0000 7ff853f9c700 1 -- start start 2023-11-04T11:23:25.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff853f9c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff84c0cee80 0x7ff84c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff853f9c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff84c15b260 0x7ff84c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff853f9c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84c0ccc70 con 0x7ff84c0ceaa0 2023-11-04T11:23:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff853f9c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84c15baa0 con 0x7ff84c15b260 2023-11-04T11:23:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff853f9c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84c0ccec0 con 0x7ff84c0cee80 2023-11-04T11:23:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff851d38700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff84c0ceaa0 0x7ff84c0cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42122/0 (socket says 172.21.15.3:42122) 2023-11-04T11:23:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff851d38700 1 -- 172.21.15.3:0/1199841643 learned_addr learned my addr 172.21.15.3:0/1199841643 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-11-04T11:23:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff852539700 1 --2- 172.21.15.3:0/1199841643 >> v2:172.21.15.3:3300/0 conn(0x7ff84c15b260 0x7ff84c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff851537700 1 --2- 172.21.15.3:0/1199841643 >> v2:172.21.15.188:3300/0 conn(0x7ff84c0cee80 0x7ff84c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff852539700 1 -- 172.21.15.3:0/1199841643 >> v2:172.21.15.188:3300/0 conn(0x7ff84c0cee80 msgr2=0x7ff84c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T11:23:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff852539700 1 --2- 172.21.15.3:0/1199841643 >> v2:172.21.15.188:3300/0 conn(0x7ff84c0cee80 0x7ff84c15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.218+0000 7ff852539700 1 -- 172.21.15.3:0/1199841643 >> v1:172.21.15.3:6789/0 conn(0x7ff84c0ceaa0 legacy=0x7ff84c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.219+0000 7ff852539700 1 -- 172.21.15.3:0/1199841643 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c0cd1a0 con 0x7ff84c15b260 2023-11-04T11:23:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.219+0000 7ff852539700 1 --2- 172.21.15.3:0/1199841643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c15b260 0x7ff84c0caff0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7ff84c06dfe0 tx=0x7ff84400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e5358d80a99ae8 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.219+0000 7ff850d36700 1 -- 172.21.15.3:0/1199841643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84400bd20 con 0x7ff84c15b260 2023-11-04T11:23:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.219+0000 7ff850d36700 1 -- 172.21.15.3:0/1199841643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84400be80 con 0x7ff84c15b260 2023-11-04T11:23:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.219+0000 7ff850d36700 1 -- 172.21.15.3:0/1199841643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440116a0 con 0x7ff84c15b260 2023-11-04T11:23:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 -- 172.21.15.3:0/1199841643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c15b260 msgr2=0x7ff84c0caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 --2- 172.21.15.3:0/1199841643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c15b260 0x7ff84c0caff0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7ff84c06dfe0 tx=0x7ff84400b330 comp rx=0 tx=0).stop 2023-11-04T11:23:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 -- 172.21.15.3:0/1199841643 shutdown_connections 2023-11-04T11:23:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 --2- 172.21.15.3:0/1199841643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c15b260 0x7ff84c0caff0 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 --2- 172.21.15.3:0/1199841643 >> v2:172.21.15.188:3300/0 conn(0x7ff84c0cee80 0x7ff84c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 -- 172.21.15.3:0/1199841643 >> 172.21.15.3:0/1199841643 conn(0x7ff84c072560 msgr2=0x7ff84c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 -- 172.21.15.3:0/1199841643 shutdown_connections 2023-11-04T11:23:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.220+0000 7ff853f9c700 1 -- 172.21.15.3:0/1199841643 wait complete. 2023-11-04T11:23:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.222+0000 7ff853f9c700 1 Processor -- start 2023-11-04T11:23:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.222+0000 7ff853f9c700 1 -- start start 2023-11-04T11:23:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff853f9c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84c0ceaa0 0x7ff84c14fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff853f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 0x7ff84c0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff853f9c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff84c15b260 0x7ff84c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff853f9c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c80f0 con 0x7ff84c0cee80 2023-11-04T11:23:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff853f9c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c9410 con 0x7ff84c15b260 2023-11-04T11:23:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff853f9c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c77c0 con 0x7ff84c0ceaa0 2023-11-04T11:23:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff851537700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 0x7ff84c0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff851537700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 0x7ff84c0c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36782/0 (socket says 172.21.15.3:36782) 2023-11-04T11:23:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff851537700 1 -- 172.21.15.3:0/3143929859 learned_addr learned my addr 172.21.15.3:0/3143929859 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff851d38700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84c0ceaa0 0x7ff84c14fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.223+0000 7ff852539700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff84c15b260 0x7ff84c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff851537700 1 -- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84c0ceaa0 msgr2=0x7ff84c14fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff851537700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84c0ceaa0 0x7ff84c14fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff851537700 1 -- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff84c15b260 msgr2=0x7ff84c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff851537700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff84c15b260 0x7ff84c156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff851537700 1 -- 172.21.15.3:0/3143929859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844009a70 con 0x7ff84c0cee80 2023-11-04T11:23:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff851537700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 0x7ff84c0c8950 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e9b0 tx=0x7ff83c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c009710 con 0x7ff84c0cee80 2023-11-04T11:23:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c00c850 con 0x7ff84c0cee80 2023-11-04T11:23:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c0c7aa0 con 0x7ff84c0cee80 2023-11-04T11:23:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00cbc0 con 0x7ff84c0cee80 2023-11-04T11:23:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.224+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c150020 con 0x7ff84c0cee80 2023-11-04T11:23:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.225+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c0cf9c0 con 0x7ff84c0cee80 2023-11-04T11:23:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.226+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff83c00cd20 con 0x7ff84c0cee80 2023-11-04T11:23:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.226+0000 7ff8427fc700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff83006af00 0x7ff83006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.227+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff83c089870 con 0x7ff84c0cee80 2023-11-04T11:23:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.227+0000 7ff851d38700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff83006af00 0x7ff83006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.227+0000 7ff851d38700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff83006af00 0x7ff83006d3c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff848005910 tx=0x7ff848009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.231+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7ff83c054730 con 0x7ff84c0cee80 2023-11-04T11:23:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.443+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff84c074df0 con 0x7ff84c0cee80 2023-11-04T11:23:25.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.443+0000 7ff8427fc700 1 -- 172.21.15.3:0/3143929859 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff83c021020 con 0x7ff84c0cee80 2023-11-04T11:23:25.445 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-11-04T11:23:25.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.445+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff83006af00 msgr2=0x7ff83006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.445+0000 7ff853f9c700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff83006af00 0x7ff83006d3c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff848005910 tx=0x7ff848009040 comp rx=0 tx=0).stop 2023-11-04T11:23:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.445+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 msgr2=0x7ff84c0c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.445+0000 7ff853f9c700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 0x7ff84c0c8950 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e9b0 tx=0x7ff83c00ecc0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 shutdown_connections 2023-11-04T11:23:25.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7ff83006af00 0x7ff83006d3c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff84c15b260 0x7ff84c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84c0ceaa0 0x7ff84c14fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 --2- 172.21.15.3:0/3143929859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff84c0cee80 0x7ff84c0c8950 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 >> 172.21.15.3:0/3143929859 conn(0x7ff84c072560 msgr2=0x7ff84c06ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:25.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 shutdown_connections 2023-11-04T11:23:25.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.446+0000 7ff853f9c700 1 -- 172.21.15.3:0/3143929859 wait complete. 2023-11-04T11:23:25.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-04T11:23:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.596+0000 7fdda3ccb700 1 Processor -- start 2023-11-04T11:23:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 -- start start 2023-11-04T11:23:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdd9c1611d0 0x7fdd9c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd9c06b060 0x7fdd9c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c0ce0a0 con 0x7fdd9c06b060 2023-11-04T11:23:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c06b4f0 con 0x7fdd9c1611d0 2023-11-04T11:23:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.597+0000 7fdda3ccb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0caca0 con 0x7fdd9c0cd4b0 2023-11-04T11:23:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd9c06b060 0x7fdd9c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdd9c06b060 0x7fdd9c0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36792/0 (socket says 172.21.15.3:36792) 2023-11-04T11:23:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 -- 172.21.15.3:0/116431180 learned_addr learned my addr 172.21.15.3:0/116431180 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda1266700 1 --2- 172.21.15.3:0/116431180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda1a67700 1 --2- 172.21.15.3:0/116431180 >> v2:172.21.15.188:3300/0 conn(0x7fdd9c1611d0 0x7fdd9c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 -- 172.21.15.3:0/116431180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 msgr2=0x7fdd9c06ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T11:23:25.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 --2- 172.21.15.3:0/116431180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c06ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 -- 172.21.15.3:0/116431180 >> v2:172.21.15.188:3300/0 conn(0x7fdd9c1611d0 msgr2=0x7fdd9c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T11:23:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 --2- 172.21.15.3:0/116431180 >> v2:172.21.15.188:3300/0 conn(0x7fdd9c1611d0 0x7fdd9c0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 -- 172.21.15.3:0/116431180 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c0cfe40 con 0x7fdd9c06b060 2023-11-04T11:23:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda1a67700 1 --2- 172.21.15.3:0/116431180 >> v2:172.21.15.188:3300/0 conn(0x7fdd9c1611d0 0x7fdd9c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T11:23:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda2268700 1 --2- 172.21.15.3:0/116431180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c06b060 0x7fdd9c0aff20 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c072aa0 tx=0x7fdd9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d179abca5017af65 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda0a65700 1 -- 172.21.15.3:0/116431180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400e070 con 0x7fdd9c06b060 2023-11-04T11:23:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda0a65700 1 -- 172.21.15.3:0/116431180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9400b910 con 0x7fdd9c06b060 2023-11-04T11:23:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.598+0000 7fdda0a65700 1 -- 172.21.15.3:0/116431180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400bcc0 con 0x7fdd9c06b060 2023-11-04T11:23:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 -- 172.21.15.3:0/116431180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c06b060 msgr2=0x7fdd9c0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/116431180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c06b060 0x7fdd9c0aff20 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c072aa0 tx=0x7fdd9400b330 comp rx=0 tx=0).stop 2023-11-04T11:23:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 -- 172.21.15.3:0/116431180 shutdown_connections 2023-11-04T11:23:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/116431180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c06b060 0x7fdd9c0aff20 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/116431180 >> v2:172.21.15.188:3300/0 conn(0x7fdd9c1611d0 0x7fdd9c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/116431180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c06ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 -- 172.21.15.3:0/116431180 >> 172.21.15.3:0/116431180 conn(0x7fdd9c072560 msgr2=0x7fdd9c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 -- 172.21.15.3:0/116431180 shutdown_connections 2023-11-04T11:23:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.599+0000 7fdda3ccb700 1 -- 172.21.15.3:0/116431180 wait complete. 2023-11-04T11:23:25.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.600+0000 7fdda3ccb700 1 Processor -- start 2023-11-04T11:23:25.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 -- start start 2023-11-04T11:23:25.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd9c06b060 0x7fdd9c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 0x7fdd9c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0c6b00 con 0x7fdd9c1611d0 2023-11-04T11:23:25.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c152fc0 con 0x7fdd9c06b060 2023-11-04T11:23:25.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda3ccb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0c5e80 con 0x7fdd9c0cd4b0 2023-11-04T11:23:25.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda2268700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 0x7fdd9c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda2268700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 0x7fdd9c15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36798/0 (socket says 172.21.15.3:36798) 2023-11-04T11:23:25.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.601+0000 7fdda2268700 1 -- 172.21.15.3:0/3123684025 learned_addr learned my addr 172.21.15.3:0/3123684025 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:25.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda1266700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda1a67700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd9c06b060 0x7fdd9c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda2268700 1 -- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 msgr2=0x7fdd9c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda2268700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda2268700 1 -- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd9c06b060 msgr2=0x7fdd9c152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda2268700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd9c06b060 0x7fdd9c152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda2268700 1 -- 172.21.15.3:0/3123684025 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd94009b30 con 0x7fdd9c1611d0 2023-11-04T11:23:25.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda1266700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T11:23:25.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda1a67700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd9c06b060 0x7fdd9c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T11:23:25.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda2268700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 0x7fdd9c15ed30 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0d1750 tx=0x7fdd94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd940092e0 con 0x7fdd9c1611d0 2023-11-04T11:23:25.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94004250 con 0x7fdd9c1611d0 2023-11-04T11:23:25.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94004600 con 0x7fdd9c1611d0 2023-11-04T11:23:25.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c0c6160 con 0x7fdd9c1611d0 2023-11-04T11:23:25.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.602+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c0c6720 con 0x7fdd9c1611d0 2023-11-04T11:23:25.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.603+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c0c6c40 con 0x7fdd9c1611d0 2023-11-04T11:23:25.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.604+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd94004760 con 0x7fdd9c1611d0 2023-11-04T11:23:25.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.604+0000 7fdd927fc700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdd8006af00 0x7fdd8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.605+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd9408afa0 con 0x7fdd9c1611d0 2023-11-04T11:23:25.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.605+0000 7fdda1a67700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdd8006af00 0x7fdd8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.605+0000 7fdda1a67700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdd8006af00 0x7fdd8006d3c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c960 tx=0x7fdd98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.609+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fdd940bde40 con 0x7fdd9c1611d0 2023-11-04T11:23:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.814+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd9c074df0 con 0x7fdd9c1611d0 2023-11-04T11:23:25.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.815+0000 7fdd927fc700 1 -- 172.21.15.3:0/3123684025 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd9c0c6c40 con 0x7fdd9c1611d0 2023-11-04T11:23:25.816 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-11-04T11:23:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdd8006af00 msgr2=0x7fdd8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdd8006af00 0x7fdd8006d3c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c960 tx=0x7fdd98008040 comp rx=0 tx=0).stop 2023-11-04T11:23:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 msgr2=0x7fdd9c15ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 0x7fdd9c15ed30 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0d1750 tx=0x7fdd94004ad0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 shutdown_connections 2023-11-04T11:23:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fdd8006af00 0x7fdd8006d3c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd9c06b060 0x7fdd9c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9c1611d0 0x7fdd9c15ed30 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 --2- 172.21.15.3:0/3123684025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9c0cd4b0 0x7fdd9c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 >> 172.21.15.3:0/3123684025 conn(0x7fdd9c072560 msgr2=0x7fdd9c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 shutdown_connections 2023-11-04T11:23:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.817+0000 7fdda3ccb700 1 -- 172.21.15.3:0/3123684025 wait complete. 2023-11-04T11:23:25.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-04T11:23:25.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.970+0000 7fd8d6972700 1 Processor -- start 2023-11-04T11:23:25.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.970+0000 7fd8d6972700 1 -- start start 2023-11-04T11:23:25.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8d6972700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8d6972700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8d00d02d0 0x7fd8d00d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8d6972700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00caca0 con 0x7fd8d01611d0 2023-11-04T11:23:25.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8d6972700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d00ce0a0 con 0x7fd8d00d02d0 2023-11-04T11:23:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8d6972700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d006b4e0 con 0x7fd8d006ab20 2023-11-04T11:23:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8cf7fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8d4f0f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8d00d02d0 0x7fd8d00d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8cf7fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d006afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:56092/0 (socket says 172.21.15.3:56092) 2023-11-04T11:23:25.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.971+0000 7fd8cf7fe700 1 -- 172.21.15.3:0/2988606707 learned_addr learned my addr 172.21.15.3:0/2988606707 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:25.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8cf7fe700 1 -- 172.21.15.3:0/2988606707 >> v2:172.21.15.188:3300/0 conn(0x7fd8d00d02d0 msgr2=0x7fd8d00d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T11:23:25.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8cf7fe700 1 --2- 172.21.15.3:0/2988606707 >> v2:172.21.15.188:3300/0 conn(0x7fd8d00d02d0 0x7fd8d00d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8cf7fe700 1 -- 172.21.15.3:0/2988606707 >> v1:172.21.15.188:6789/0 conn(0x7fd8d01611d0 legacy=0x7fd8d00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8cf7fe700 1 -- 172.21.15.3:0/2988606707 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00cfe40 con 0x7fd8d006ab20 2023-11-04T11:23:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8d4f0f700 1 --2- 172.21.15.3:0/2988606707 >> v2:172.21.15.188:3300/0 conn(0x7fd8d00d02d0 0x7fd8d00d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T11:23:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8cf7fe700 1 --2- 172.21.15.3:0/2988606707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d006afa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40098f0 tx=0x7fd8c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0820d6e927b3647 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8ceffd700 1 -- 172.21.15.3:0/2988606707 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4011830 con 0x7fd8d006ab20 2023-11-04T11:23:25.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8ceffd700 1 -- 172.21.15.3:0/2988606707 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c4011990 con 0x7fd8d006ab20 2023-11-04T11:23:25.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.972+0000 7fd8ceffd700 1 -- 172.21.15.3:0/2988606707 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4011d20 con 0x7fd8d006ab20 2023-11-04T11:23:25.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 -- 172.21.15.3:0/2988606707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 msgr2=0x7fd8d006afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2988606707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d006afa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40098f0 tx=0x7fd8c4009c00 comp rx=0 tx=0).stop 2023-11-04T11:23:25.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 -- 172.21.15.3:0/2988606707 shutdown_connections 2023-11-04T11:23:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2988606707 >> v2:172.21.15.188:3300/0 conn(0x7fd8d00d02d0 0x7fd8d00d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2988606707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d006afa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 -- 172.21.15.3:0/2988606707 >> 172.21.15.3:0/2988606707 conn(0x7fd8d0072560 msgr2=0x7fd8d0099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 -- 172.21.15.3:0/2988606707 shutdown_connections 2023-11-04T11:23:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.973+0000 7fd8d6972700 1 -- 172.21.15.3:0/2988606707 wait complete. 2023-11-04T11:23:25.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.974+0000 7fd8d6972700 1 Processor -- start 2023-11-04T11:23:25.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.974+0000 7fd8d6972700 1 -- start start 2023-11-04T11:23:25.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8d6972700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d015ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8d6972700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00d02d0 0x7fd8d00b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8d6972700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8d01611d0 0x7fd8d00cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8d6972700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c7070 con 0x7fd8d00d02d0 2023-11-04T11:23:25.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8d6972700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c67d0 con 0x7fd8d01611d0 2023-11-04T11:23:25.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8d6972700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c6ab0 con 0x7fd8d006ab20 2023-11-04T11:23:25.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.975+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d015ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d015ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.3:56096/0 (socket says 172.21.15.3:56096) 2023-11-04T11:23:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 -- 172.21.15.3:0/2675252712 learned_addr learned my addr 172.21.15.3:0/2675252712 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8d4f0f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8d01611d0 0x7fd8d00cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cf7fe700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00d02d0 0x7fd8d00b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 -- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8d01611d0 msgr2=0x7fd8d00cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8d01611d0 0x7fd8d00cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 -- 172.21.15.3:0/2675252712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00d02d0 msgr2=0x7fd8d00b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00d02d0 0x7fd8d00b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:25.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 -- 172.21.15.3:0/2675252712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8009a70 con 0x7fd8d006ab20 2023-11-04T11:23:25.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8d4f0f700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8d01611d0 0x7fd8d00cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-04T11:23:25.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8cffff700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d015ffd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c450 tx=0x7fd8c000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00052d0 con 0x7fd8d006ab20 2023-11-04T11:23:25.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0005430 con 0x7fd8d006ab20 2023-11-04T11:23:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00038c0 con 0x7fd8d006ab20 2023-11-04T11:23:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c4009550 con 0x7fd8d006ab20 2023-11-04T11:23:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.976+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d015f7d0 con 0x7fd8d006ab20 2023-11-04T11:23:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.978+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d0160510 con 0x7fd8d006ab20 2023-11-04T11:23:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.978+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8c0003a20 con 0x7fd8d006ab20 2023-11-04T11:23:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.979+0000 7fd8ccff9700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8bc06af00 0x7fd8bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.979+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8c00889a0 con 0x7fd8d006ab20 2023-11-04T11:23:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.979+0000 7fd8cf7fe700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8bc06af00 0x7fd8bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.980+0000 7fd8cf7fe700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8bc06af00 0x7fd8bc06d3c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009c40 tx=0x7fd8c4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:25.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:25.984+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7fd8c0053980 con 0x7fd8d006ab20 2023-11-04T11:23:26.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.185+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8d015f630 con 0x7fd8d006ab20 2023-11-04T11:23:26.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.186+0000 7fd8ccff9700 1 -- 172.21.15.3:0/2675252712 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8d015f630 con 0x7fd8d006ab20 2023-11-04T11:23:26.187 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-11-04T11:23:26.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8bc06af00 msgr2=0x7fd8bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8bc06af00 0x7fd8bc06d3c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009c40 tx=0x7fd8c4008030 comp rx=0 tx=0).stop 2023-11-04T11:23:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 msgr2=0x7fd8d015ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d015ffd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c450 tx=0x7fd8c000c760 comp rx=0 tx=0).stop 2023-11-04T11:23:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 shutdown_connections 2023-11-04T11:23:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7fd8bc06af00 0x7fd8bc06d3c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8d01611d0 0x7fd8d00cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8d006ab20 0x7fd8d015ffd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 --2- 172.21.15.3:0/2675252712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8d00d02d0 0x7fd8d00b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 >> 172.21.15.3:0/2675252712 conn(0x7fd8d0072560 msgr2=0x7fd8d00cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 shutdown_connections 2023-11-04T11:23:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.189+0000 7fd8d6972700 1 -- 172.21.15.3:0/2675252712 wait complete. 2023-11-04T11:23:26.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-04T11:23:26.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.344+0000 7f6e5b0f4700 1 Processor -- start 2023-11-04T11:23:26.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.344+0000 7f6e5b0f4700 1 -- start start 2023-11-04T11:23:26.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e5b0f4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e5406ad90 0x7f6e540b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:26.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e5b0f4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e540ce130 0x7f6e540cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:26.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e5b0f4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540b1ce0 con 0x7f6e540cdb30 2023-11-04T11:23:26.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e5b0f4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540ccd70 con 0x7f6e540ce130 2023-11-04T11:23:26.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e5b0f4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540cd050 con 0x7f6e5406ad90 2023-11-04T11:23:26.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e59691700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e540ce130 0x7f6e540cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:26.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e59691700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6e540ce130 0x7f6e540cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36820/0 (socket says 172.21.15.3:36820) 2023-11-04T11:23:26.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e59691700 1 -- 172.21.15.3:0/1506608352 learned_addr learned my addr 172.21.15.3:0/1506608352 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:26.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.345+0000 7f6e58e90700 1 --2- 172.21.15.3:0/1506608352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e5406ad90 0x7f6e540b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:26.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e59691700 1 -- 172.21.15.3:0/1506608352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e5406ad90 msgr2=0x7f6e540b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-04T11:23:26.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e59691700 1 --2- 172.21.15.3:0/1506608352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e5406ad90 0x7f6e540b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e59691700 1 -- 172.21.15.3:0/1506608352 >> v1:172.21.15.188:6789/0 conn(0x7f6e540cdb30 legacy=0x7f6e540cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e59691700 1 -- 172.21.15.3:0/1506608352 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540cd330 con 0x7f6e540ce130 2023-11-04T11:23:26.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e59691700 1 --2- 172.21.15.3:0/1506608352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e540ce130 0x7f6e540cb2c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e50 tx=0x7f6e4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa8b91ceb8cfbb server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:26.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e537fe700 1 -- 172.21.15.3:0/1506608352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00e070 con 0x7f6e540ce130 2023-11-04T11:23:26.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e537fe700 1 -- 172.21.15.3:0/1506608352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c00b910 con 0x7f6e540ce130 2023-11-04T11:23:26.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.346+0000 7f6e537fe700 1 -- 172.21.15.3:0/1506608352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00bcc0 con 0x7f6e540ce130 2023-11-04T11:23:26.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/1506608352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e540ce130 msgr2=0x7f6e540cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/1506608352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e540ce130 0x7f6e540cb2c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e50 tx=0x7f6e4c00b330 comp rx=0 tx=0).stop 2023-11-04T11:23:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/1506608352 shutdown_connections 2023-11-04T11:23:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/1506608352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e540ce130 0x7f6e540cb2c0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/1506608352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e5406ad90 0x7f6e540b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/1506608352 >> 172.21.15.3:0/1506608352 conn(0x7f6e540d0560 msgr2=0x7f6e540d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/1506608352 shutdown_connections 2023-11-04T11:23:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.347+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/1506608352 wait complete. 2023-11-04T11:23:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 Processor -- start 2023-11-04T11:23:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 -- start start 2023-11-04T11:23:26.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 0x7f6e5414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:26.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e540cdb30 0x7f6e5414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:26.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e540ce130 0x7f6e540c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:26.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540c9e50 con 0x7f6e5406ad90 2023-11-04T11:23:26.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5414cf00 con 0x7f6e540cdb30 2023-11-04T11:23:26.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e5b0f4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540c9b70 con 0x7f6e540ce130 2023-11-04T11:23:26.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e58e90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 0x7f6e5414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:26.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.349+0000 7f6e58e90700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 0x7f6e5414d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36822/0 (socket says 172.21.15.3:36822) 2023-11-04T11:23:26.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 -- 172.21.15.3:0/3604840813 learned_addr learned my addr 172.21.15.3:0/3604840813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-11-04T11:23:26.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e59691700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e540ce130 0x7f6e540c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:26.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e53fff700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e540cdb30 0x7f6e5414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:26.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 -- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e540ce130 msgr2=0x7f6e540c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e540ce130 0x7f6e540c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 -- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e540cdb30 msgr2=0x7f6e5414c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e540cdb30 0x7f6e5414c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 -- 172.21.15.3:0/3604840813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c009b30 con 0x7f6e5406ad90 2023-11-04T11:23:26.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e53fff700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e540cdb30 0x7f6e5414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-04T11:23:26.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e58e90700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 0x7f6e5414d1e0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800e930 tx=0x7f6e4800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:26.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48009650 con 0x7f6e5406ad90 2023-11-04T11:23:26.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4800ee80 con 0x7f6e5406ad90 2023-11-04T11:23:26.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e540c77d0 con 0x7f6e5406ad90 2023-11-04T11:23:26.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800c9f0 con 0x7f6e5406ad90 2023-11-04T11:23:26.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.350+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540c7d90 con 0x7f6e5406ad90 2023-11-04T11:23:26.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.352+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540c7610 con 0x7f6e5406ad90 2023-11-04T11:23:26.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.352+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e4800cb50 con 0x7f6e5406ad90 2023-11-04T11:23:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.353+0000 7f6e517fa700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6e3806af30 0x7f6e3806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-04T11:23:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.353+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e480899c0 con 0x7f6e5406ad90 2023-11-04T11:23:26.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.353+0000 7f6e53fff700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6e3806af30 0x7f6e3806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-04T11:23:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.354+0000 7f6e53fff700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6e3806af30 0x7f6e3806d3f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6e5406ec90 tx=0x7f6e40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-04T11:23:26.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.358+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179750 (secure 0 0 0) 0x7f6e48054910 con 0x7f6e5406ad90 2023-11-04T11:23:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.581+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e54074df0 con 0x7f6e5406ad90 2023-11-04T11:23:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.582+0000 7f6e517fa700 1 -- 172.21.15.3:0/3604840813 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e48021080 con 0x7f6e5406ad90 2023-11-04T11:23:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6e3806af30 msgr2=0x7f6e3806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6e3806af30 0x7f6e3806d3f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6e5406ec90 tx=0x7f6e40009040 comp rx=0 tx=0).stop 2023-11-04T11:23:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 msgr2=0x7f6e5414d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-04T11:23:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 0x7f6e5414d1e0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800e930 tx=0x7f6e4800ec40 comp rx=0 tx=0).stop 2023-11-04T11:23:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 shutdown_connections 2023-11-04T11:23:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:6832/3102759174,v1:172.21.15.188:6833/3102759174] conn(0x7f6e3806af30 0x7f6e3806d3f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e540ce130 0x7f6e540c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e5406ad90 0x7f6e5414d1e0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 --2- 172.21.15.3:0/3604840813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e540cdb30 0x7f6e5414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-04T11:23:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 >> 172.21.15.3:0/3604840813 conn(0x7f6e540d0560 msgr2=0x7f6e540cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-04T11:23:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 shutdown_connections 2023-11-04T11:23:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-11-04T11:23:26.584+0000 7f6e5b0f4700 1 -- 172.21.15.3:0/3604840813 wait complete. 2023-11-04T11:23:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-11-04T11:23:26.599 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-04T11:23:26.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T11:23:26.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-04T11:23:26.677 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T11:23:26.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T11:23:26.678 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-04T11:23:26.737 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-11-04T11:23:26.737 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T11:23:26.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16909 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 172.21.15.3:6789:/test /tmp/tmp.Ny4doz2FAUtest ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb,acl 0 0 172.21.15.3:6789:/scratch /tmp/tmp.WC1AnNZjMJscratch ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=7a64e299-05b0-4fd0-819f-2262a06846bb,acl 0 0 2023-11-04T11:23:26.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-11-04T11:23:26.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T11:23:26.757 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-04T11:23:26.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T11:23:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-04T11:23:26.816 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T11:23:26.816 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T11:23:26.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2970 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-04T11:23:26.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T11:23:26.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T11:23:26.851 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-04T11:23:26.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T11:23:26.913 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-04T11:23:26.914 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-04T11:23:26.914 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-04T11:23:26.974 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2970 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-04T11:23:26.974 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-04T11:23:26.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-04T11:23:26.975 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-04T11:23:27.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-04T11:23:27.032 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-04T11:23:28.642 INFO:tasks.ceph.osd.0:Sent signal 15 2023-11-04T11:23:28.643 INFO:tasks.ceph.osd.1:Sent signal 15 2023-11-04T11:23:28.643 INFO:tasks.ceph.osd.3:Sent signal 15 2023-11-04T11:23:28.644 INFO:tasks.ceph.osd.4:Sent signal 15 2023-11-04T11:23:28.644 INFO:tasks.ceph.osd.5:Sent signal 15 2023-11-04T11:23:28.644 INFO:tasks.ceph.osd.6:Sent signal 15 2023-11-04T11:23:28.645 INFO:tasks.ceph.osd.7:Sent signal 15 2023-11-04T11:23:28.645 INFO:tasks.ceph.mds.a:Sent signal 15 2023-11-04T11:23:28.645 INFO:tasks.ceph.mds.c:Sent signal 15 2023-11-04T11:23:28.646 INFO:tasks.ceph.mds.b:Sent signal 15 2023-11-04T11:23:28.646 INFO:tasks.ceph.mds.d:Sent signal 15 2023-11-04T11:23:28.647 INFO:tasks.ceph.mon.a:Sent signal 15 2023-11-04T11:23:28.647 INFO:tasks.ceph.mon.b:Sent signal 15 2023-11-04T11:23:28.647 INFO:tasks.ceph.mon.c:Sent signal 15 2023-11-04T11:23:28.648 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-11-04T11:23:28.648 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-11-04T11:23:28.649 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:23:28.643+0000 7eff274c1700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 104403) UID: 0 2023-11-04T11:23:28.649 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:23:28.643+0000 7eff274c1700 -1 osd.4 44 *** Got signal Terminated *** 2023-11-04T11:23:28.649 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-04T11:23:28.643+0000 7eff274c1700 -1 osd.4 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.650 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:23:28.643+0000 7f1044bc0700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 104406) UID: 0 2023-11-04T11:23:28.650 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:23:28.643+0000 7f1044bc0700 -1 osd.5 44 *** Got signal Terminated *** 2023-11-04T11:23:28.650 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-04T11:23:28.643+0000 7f1044bc0700 -1 osd.5 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.651 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:23:28.644+0000 7f8a2900f700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 104409) UID: 0 2023-11-04T11:23:28.651 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:23:28.644+0000 7f8a2900f700 -1 osd.6 44 *** Got signal Terminated *** 2023-11-04T11:23:28.651 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-04T11:23:28.644+0000 7f8a2900f700 -1 osd.6 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.652 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:23:28.644+0000 7fdadcd9d700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 104411) UID: 0 2023-11-04T11:23:28.652 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:23:28.644+0000 7fdadcd9d700 -1 osd.7 44 *** Got signal Terminated *** 2023-11-04T11:23:28.652 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-04T11:23:28.644+0000 7fdadcd9d700 -1 osd.7 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.653 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T11:23:28.643+0000 7f58ab082700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 104038) UID: 0 2023-11-04T11:23:28.653 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T11:23:28.643+0000 7f58ab082700 -1 osd.0 44 *** Got signal Terminated *** 2023-11-04T11:23:28.653 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-11-04T11:23:28.643+0000 7f58ab082700 -1 osd.0 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.654 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T11:23:28.643+0000 7f7dba50c700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 104043) UID: 0 2023-11-04T11:23:28.654 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T11:23:28.643+0000 7f7dba50c700 -1 osd.1 44 *** Got signal Terminated *** 2023-11-04T11:23:28.654 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-11-04T11:23:28.643+0000 7f7dba50c700 -1 osd.1 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.655 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T11:23:28.644+0000 7f97f5987700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 104047) UID: 0 2023-11-04T11:23:28.655 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T11:23:28.644+0000 7f97f5987700 -1 osd.3 44 *** Got signal Terminated *** 2023-11-04T11:23:28.655 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-11-04T11:23:28.644+0000 7f97f5987700 -1 osd.3 44 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-04T11:23:28.656 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T11:23:28.645+0000 7fda51433700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 119092) UID: 0 2023-11-04T11:23:28.656 INFO:tasks.ceph.mds.a.smithi003.stderr:2023-11-04T11:23:28.645+0000 7fda51433700 -1 mds.a *** got signal Terminated *** 2023-11-04T11:23:28.656 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T11:23:28.646+0000 7f0aafeef700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 119093) UID: 0 2023-11-04T11:23:28.656 INFO:tasks.ceph.mds.c.smithi003.stderr:2023-11-04T11:23:28.646+0000 7f0aafeef700 -1 mds.c *** got signal Terminated *** 2023-11-04T11:23:28.657 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-11-04T11:23:28.647+0000 7f99ef92b700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 103720) UID: 0 2023-11-04T11:23:28.657 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-11-04T11:23:28.647+0000 7f99ef92b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-04T11:23:28.658 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-04T11:23:28.648+0000 7f25cf230700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 111550) UID: 0 2023-11-04T11:23:28.658 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f25cf230700 -1 mds.b *** got signal Terminated *** 2023-11-04T11:23:28.659 INFO:tasks.ceph.mon.b.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f9e85ed6700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 103752) UID: 0 2023-11-04T11:23:28.659 INFO:tasks.ceph.mon.b.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f9e85ed6700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-11-04T11:23:28.659 INFO:tasks.ceph.mon.c.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f9e668ab700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 103753) UID: 0 2023-11-04T11:23:28.660 INFO:tasks.ceph.mon.c.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f9e668ab700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-11-04T11:23:28.660 INFO:tasks.ceph.mds.d.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f38b2d24700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 111549) UID: 0 2023-11-04T11:23:28.660 INFO:tasks.ceph.mds.d.smithi188.stderr:2023-11-04T11:23:28.657+0000 7f38b2d24700 -1 mds.d *** got signal Terminated *** 2023-11-04T11:23:28.670 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-11-04T11:23:28.670+0000 7f99ea120700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-04T11:23:28.849 INFO:tasks.ceph.mgr.y.smithi003.stderr:daemon-helper: command crashed with signal 15 2023-11-04T11:25:53.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:25:53.867 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:26:24.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:26:24.040 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:26:54.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:26:54.200 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:27:24.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:27:24.339 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:27:54.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:27:54.479 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:28:24.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:28:24.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:28:54.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:28:54.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:29:24.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:29:24.910 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:29:55.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:29:55.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:30:25.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:30:25.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:30:55.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:30:55.325 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:31:25.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:31:25.452 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:31:55.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:31:55.578 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:32:25.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:32:25.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:32:55.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:32:55.875 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:33:26.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:33:26.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:33:56.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:33:56.120 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:34:26.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:34:26.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:34:56.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:34:56.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:35:26.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:35:26.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:35:56.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:35:56.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:36:26.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:36:26.805 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:36:56.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:36:56.965 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:37:27.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:37:27.101 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:37:57.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:37:57.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:38:27.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:38:27.364 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:38:57.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:38:57.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:39:27.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:39:27.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:39:57.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:39:57.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:40:27.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:40:27.880 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:40:57.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:40:57.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:41:28.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:41:28.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:41:58.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:41:58.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:42:28.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:42:28.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:42:58.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:42:58.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:43:28.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:43:28.684 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:43:58.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:43:58.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:44:28.984 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:44:28.988 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:44:59.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:44:59.132 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:45:29.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:45:29.234 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:45:59.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:45:59.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:46:29.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:46:29.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:46:59.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:46:59.565 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:47:29.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:47:29.700 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:47:59.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:47:59.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:48:30.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:48:30.012 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:49:00.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:49:00.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:49:30.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:49:30.374 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:50:00.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:50:00.502 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:50:30.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:50:30.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:51:00.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:51:00.797 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:51:30.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:51:30.948 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:52:01.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:52:01.093 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:52:31.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:52:31.228 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:53:01.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:53:01.379 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:53:31.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:53:31.541 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:54:01.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:54:01.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:54:31.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:54:31.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:55:01.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:55:01.946 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:55:32.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:55:32.091 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:56:02.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:56:02.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:56:32.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:56:32.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:57:02.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:57:02.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:57:32.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:57:32.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:58:02.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:58:02.834 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:58:32.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:58:32.986 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:59:03.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:59:03.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:59:33.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T11:59:33.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:00:03.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:00:03.425 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:00:33.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:00:33.568 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:01:03.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:01:03.712 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:01:33.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:01:33.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:02:04.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:02:04.100 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:02:34.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:02:34.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:03:04.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:03:04.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:03:34.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:03:34.472 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:04:04.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:04:04.607 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:04:34.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:04:34.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:05:04.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:05:04.895 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:05:35.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:05:35.147 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:06:05.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:06:05.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:06:35.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:06:35.461 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:07:05.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:07:05.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:07:35.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:07:35.733 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:08:05.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:08:05.866 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:08:36.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:08:36.034 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:09:06.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:09:06.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:09:36.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:09:36.339 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:10:06.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:10:06.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:10:36.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:10:36.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:11:06.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:11:06.762 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:11:36.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:11:36.913 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:12:07.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:12:07.054 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:12:37.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:12:37.193 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:13:07.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:13:07.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:13:37.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:13:37.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:14:07.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:14:07.648 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:14:37.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:14:37.792 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:15:07.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:15:07.928 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:15:38.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:15:38.079 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:16:08.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:16:08.240 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:16:38.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:16:38.394 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:17:08.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:17:08.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:17:38.642 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:17:38.646 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:18:08.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:18:08.781 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:18:38.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:18:38.951 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:19:09.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:19:09.081 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:19:39.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:19:39.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:20:09.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:20:09.366 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:20:39.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:20:39.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:21:09.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:21:09.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:21:39.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:21:39.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:22:09.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:22:09.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:22:40.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:22:40.008 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:23:10.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:23:10.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:23:40.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:23:40.269 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:24:10.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:24:10.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:24:40.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:24:40.542 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:25:10.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:25:10.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:25:40.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:25:40.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:26:10.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:26:10.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:26:41.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:26:41.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:27:11.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:27:11.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:27:41.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:27:41.403 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:28:11.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:28:11.547 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:28:41.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:28:41.657 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:29:11.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:29:11.801 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:29:41.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:29:41.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:30:12.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:30:12.096 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:30:42.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:30:42.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:31:12.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:31:12.367 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:31:42.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:31:42.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:32:12.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:32:12.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:32:42.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:32:42.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:33:12.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:33:12.951 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:33:43.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:33:43.103 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:34:13.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:34:13.231 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:34:43.310 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:34:43.314 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:35:13.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:35:13.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:35:43.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:35:43.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:36:13.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:36:13.772 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:36:43.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:36:43.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:37:14.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:37:14.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:37:44.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:37:44.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:38:14.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:38:14.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:38:44.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:38:44.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:39:14.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:39:14.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:39:44.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:39:44.703 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:40:14.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:40:14.837 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:40:44.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:40:44.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:41:15.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:41:15.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:41:45.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:41:45.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:42:15.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:42:15.346 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:42:45.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:42:45.466 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:43:15.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:43:15.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:43:45.706 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:43:45.710 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:44:15.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:44:15.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:44:45.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:44:45.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:45:16.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:45:16.117 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:45:46.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:45:46.269 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:46:16.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:46:16.413 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:46:46.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:46:46.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:47:16.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:47:16.716 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:47:46.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:47:46.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:48:16.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:48:16.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:48:47.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:48:47.047 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:49:17.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:49:17.187 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:49:47.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:49:47.318 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:50:17.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:50:17.461 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:50:47.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:50:47.599 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:51:17.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:51:17.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:51:47.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:51:47.860 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:52:17.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:52:17.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:52:48.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:52:48.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:53:18.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:53:18.290 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:53:48.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:53:48.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:54:18.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:54:18.568 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:54:48.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:54:48.704 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:55:18.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:55:18.848 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:55:49.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:55:49.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:56:19.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:56:19.161 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:56:49.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:56:49.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:57:19.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:57:19.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:57:49.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:57:49.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:58:19.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:58:19.769 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:58:49.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:58:49.904 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:59:20.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:59:20.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:59:50.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T12:59:50.200 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:00:20.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:00:20.344 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:00:50.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:00:50.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:01:20.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:01:20.624 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:01:50.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:01:50.776 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:02:20.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:02:20.927 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:02:51.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:02:51.078 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:03:21.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:03:21.223 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:03:51.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:03:51.375 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:04:21.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:04:21.519 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:04:51.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:04:51.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:05:21.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:05:21.814 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:05:51.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:05:51.985 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:06:22.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:06:22.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:06:52.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:06:52.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:07:22.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:07:22.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:07:52.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:07:52.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:08:22.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:08:22.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:08:52.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:08:52.811 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:09:22.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:09:22.972 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:09:53.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:09:53.110 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:10:23.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:10:23.226 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:10:53.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:10:53.379 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:11:23.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:11:23.521 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:11:53.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:11:53.665 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:12:23.780 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:12:23.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:12:53.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:12:53.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:13:24.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:13:24.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:13:54.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:13:54.164 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:14:24.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:14:24.309 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:14:54.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:14:54.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:15:24.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:15:24.612 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:15:54.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:15:54.740 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:16:24.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:16:24.883 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:16:55.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:16:55.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:17:25.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:17:25.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:17:55.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:17:55.348 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:18:25.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:18:25.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:18:55.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:18:55.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:19:25.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:19:25.687 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:19:55.827 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:19:55.831 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:20:25.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:20:25.974 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:20:56.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:20:56.110 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:21:26.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:21:26.228 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:21:56.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:21:56.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:22:26.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:22:26.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:22:56.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:22:56.668 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:23:26.808 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:23:26.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:23:56.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:23:56.961 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:24:27.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:24:27.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:24:57.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:24:57.242 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:25:27.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:25:27.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:25:57.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:25:57.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:26:27.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:26:27.624 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:26:57.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:26:57.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:27:27.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:27:27.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:27:58.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:27:58.071 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:28:28.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:28:28.207 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:28:58.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:28:58.355 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:29:28.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:29:28.486 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:29:58.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:29:58.629 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:30:28.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:30:28.699 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:30:58.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:30:58.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:31:28.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:31:28.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:31:59.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:31:59.130 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:32:29.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:32:29.265 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:32:59.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:32:59.404 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:33:29.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:33:29.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:33:59.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:33:59.670 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:34:29.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:34:29.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:34:59.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:34:59.967 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:35:30.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:35:30.077 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:36:00.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:36:00.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:36:30.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:36:30.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:37:00.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:37:00.541 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:37:30.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:37:30.668 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:38:00.808 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:38:00.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:38:30.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:38:30.931 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:39:01.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:39:01.074 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:39:31.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:39:31.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:40:01.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:40:01.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:40:31.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:40:31.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:41:01.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:41:01.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:41:31.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:41:31.778 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:42:01.925 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:42:01.928 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:42:32.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:42:32.088 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:43:02.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:43:02.265 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:43:32.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:43:32.409 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:44:02.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:44:02.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:44:32.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:44:32.663 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:45:02.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:45:02.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:45:32.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:45:32.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:46:03.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:46:03.053 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:46:33.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:46:33.197 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:47:03.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:47:03.357 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:47:33.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:47:33.432 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:48:03.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:48:03.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:48:33.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:48:33.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:49:03.878 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:49:03.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:49:33.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:49:33.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:50:04.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:50:04.121 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:50:34.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:50:34.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:51:04.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:51:04.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:51:34.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:51:34.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:52:04.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:52:04.642 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:52:34.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:52:34.772 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:53:04.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:53:04.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:53:35.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:53:35.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:54:05.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:54:05.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:54:35.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:54:35.356 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:55:05.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:55:05.492 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:55:35.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:55:35.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:56:05.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:56:05.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:56:35.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:56:35.906 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:57:06.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:57:06.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:57:36.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:57:36.201 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:58:06.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:58:06.320 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:58:36.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:58:36.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:59:06.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:59:06.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:59:36.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T13:59:36.718 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:00:06.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:00:06.837 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:00:36.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:00:36.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:01:07.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:01:07.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:01:37.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:01:37.252 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:02:07.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:02:07.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:02:37.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:02:37.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:03:07.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:03:07.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:03:37.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:03:37.835 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:04:07.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:04:07.970 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:04:38.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:04:38.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:05:08.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:05:08.248 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:05:38.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:05:38.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:06:08.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:06:08.557 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:06:38.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:06:38.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:07:08.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:07:08.874 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:07:39.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:07:39.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:08:09.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:08:09.144 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:08:39.254 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:08:39.258 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:09:09.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:09:09.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:09:39.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:09:39.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:10:09.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:10:09.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:10:39.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:10:39.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:11:09.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:11:09.973 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:11:40.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:11:40.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:12:10.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:12:10.286 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:12:40.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:12:40.404 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:13:10.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:13:10.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:13:40.715 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:13:40.718 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:14:10.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:14:10.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:14:41.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:14:41.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:15:11.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:15:11.158 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:15:41.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:15:41.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:16:11.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:16:11.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:16:41.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:16:41.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:17:11.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:17:11.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:17:41.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:17:41.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:18:11.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:18:12.003 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:18:42.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:18:42.155 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:19:12.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:19:12.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:19:42.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:19:42.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:20:12.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:20:12.552 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:20:42.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:20:42.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:21:12.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:21:12.865 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:21:43.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:21:43.012 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:22:13.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:22:13.153 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:22:43.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:22:43.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:23:13.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:23:13.356 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:23:43.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:23:43.503 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:24:13.648 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:24:13.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:24:43.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:24:43.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:25:13.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:25:13.948 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:25:44.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:25:44.091 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:26:14.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:26:14.226 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:26:44.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:26:44.363 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:27:14.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:27:14.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:27:44.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:27:44.658 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:28:14.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:28:14.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:28:44.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:28:44.936 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:29:15.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:29:15.088 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:29:45.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:29:45.232 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:30:15.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:30:15.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:30:45.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:30:45.529 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:31:15.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:31:15.646 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:31:45.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:31:45.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:32:15.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:32:15.942 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:32:46.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:32:46.072 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:33:16.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:33:16.196 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:33:46.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:33:46.323 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:34:16.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:34:16.459 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:34:46.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:34:46.594 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:35:16.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:35:16.713 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:35:46.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:35:46.866 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:36:16.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:36:16.999 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:36:47.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:36:47.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:37:17.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:37:17.304 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:37:47.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:37:47.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:38:17.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:38:17.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:38:47.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:38:47.727 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:39:17.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:39:17.887 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:39:48.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:39:48.022 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:40:18.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:40:18.167 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:40:48.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:40:48.285 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:41:18.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:41:18.421 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:41:48.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:41:48.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:42:18.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:42:18.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:42:48.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:42:48.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:43:18.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:43:18.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:43:49.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:43:49.121 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:44:19.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:44:19.274 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:44:49.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:44:49.342 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:45:19.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:45:19.479 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:45:49.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:45:49.629 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:46:19.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:46:19.748 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:46:49.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:46:49.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:47:20.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:47:20.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:47:50.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:47:50.187 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:48:20.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:48:20.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:48:50.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:48:50.492 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:49:20.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:49:20.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:49:50.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:49:50.763 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:50:20.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:50:20.898 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:50:51.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:50:51.055 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:51:21.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:51:21.193 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:51:51.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:51:51.338 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:52:21.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:52:21.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:52:51.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:52:51.658 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:53:21.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:53:21.810 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:53:51.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:53:51.957 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:54:22.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:54:22.097 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:54:52.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:54:52.249 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:55:22.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:55:22.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:55:52.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:55:52.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:56:22.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:56:22.673 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:56:52.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:56:52.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:57:22.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:57:22.942 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:57:53.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:57:53.111 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:58:23.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:58:23.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:58:53.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:58:53.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:59:23.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:59:23.543 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:59:53.673 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T14:59:53.677 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:00:23.841 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:00:23.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:00:53.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:00:53.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:01:24.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:01:24.133 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:01:54.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:01:54.278 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:02:24.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:02:24.454 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:02:54.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:02:54.656 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:03:24.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:03:24.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:03:54.940 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:03:54.943 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:04:25.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:04:25.095 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:04:55.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:04:55.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:05:25.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:05:25.408 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:05:55.472 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:05:55.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:06:25.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:06:25.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:06:55.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:06:55.713 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:07:25.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:07:25.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:07:55.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:07:55.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:08:26.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:08:26.127 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:08:56.267 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:08:56.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:09:26.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:09:26.390 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:09:56.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:09:56.533 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:10:26.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:10:26.896 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:10:57.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:10:57.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:11:27.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:11:27.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:11:57.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:11:57.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:12:27.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:12:27.486 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:12:57.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:12:57.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:13:27.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:13:27.780 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:13:57.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:13:57.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:14:28.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:14:28.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:14:58.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:14:58.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:15:28.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:15:28.273 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:15:58.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:15:58.340 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:16:28.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:16:28.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:16:58.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:16:58.636 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:17:28.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:17:28.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:17:58.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:17:58.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:18:29.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:18:29.101 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:18:59.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:18:59.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:19:29.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:19:29.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:19:59.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:19:59.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:20:29.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:20:29.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:20:59.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:20:59.796 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:21:29.925 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:21:29.929 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:22:00.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:22:00.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:22:30.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:22:30.225 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:23:00.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:23:00.361 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:23:30.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:23:30.487 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:24:00.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:24:00.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:24:30.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:24:30.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:25:00.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:25:00.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:25:31.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:25:31.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:26:01.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:26:01.172 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:26:31.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:26:31.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:27:01.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:27:01.425 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:27:31.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:27:31.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:28:01.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:28:01.697 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:28:31.828 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:28:31.832 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:29:01.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:29:01.976 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:29:32.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:29:32.138 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:30:02.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:30:02.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:30:32.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:30:32.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:31:02.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:31:02.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:31:32.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:31:32.702 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:32:02.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:32:02.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:32:32.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:32:32.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:33:03.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:33:03.101 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:33:33.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:33:33.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:34:03.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:34:03.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:34:33.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:34:33.524 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:35:03.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:35:03.649 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:35:33.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:35:33.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:36:03.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:36:03.887 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:36:34.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:36:34.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:37:04.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:37:04.132 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:37:34.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:37:34.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:38:04.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:38:04.445 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:38:34.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:38:34.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:39:04.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:39:04.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:39:34.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:39:34.858 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:40:04.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:40:05.002 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:40:35.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:40:35.155 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:41:05.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:41:05.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:41:35.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:41:35.441 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:42:05.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:42:05.594 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:42:35.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:42:35.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:43:05.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:43:05.898 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:43:36.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:43:36.041 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:44:06.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:44:06.167 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:44:36.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:44:36.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:45:06.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:45:06.438 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:45:36.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:45:36.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:46:06.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:46:06.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:46:36.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:46:36.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:47:06.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:47:06.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:47:37.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:47:37.108 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:48:07.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:48:07.242 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:48:37.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:48:37.386 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:49:07.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:49:07.521 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:49:37.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:49:37.648 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:50:07.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:50:07.818 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:50:37.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:50:37.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:51:08.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:51:08.100 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:51:38.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:51:38.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:52:08.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:52:08.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:52:38.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:52:38.602 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:53:08.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:53:08.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:53:38.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:53:38.898 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:54:09.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:54:09.050 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:54:39.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:54:39.209 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:55:09.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:55:09.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:55:39.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:55:39.514 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:56:09.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:56:09.575 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:56:39.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:56:39.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:57:09.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:57:09.853 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:57:40.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:57:40.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:58:10.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:58:10.132 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:58:40.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:58:40.338 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:59:10.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:59:10.470 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:59:40.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T15:59:40.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:00:10.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:00:10.740 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:00:40.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:00:40.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:01:11.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:01:11.079 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:01:41.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:01:41.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:02:11.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:02:11.340 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:02:41.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:02:41.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:03:11.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:03:11.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:03:41.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:03:41.779 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:04:11.910 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:04:11.915 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:04:42.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:04:42.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:05:12.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:05:12.172 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:05:42.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:05:42.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:06:12.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:06:12.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:06:42.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:06:42.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:07:12.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:07:12.751 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:07:42.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:07:42.871 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:08:13.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:08:13.022 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:08:43.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:08:43.185 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:09:13.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:09:13.351 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:09:43.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:09:43.511 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:10:13.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:10:13.639 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:10:43.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:10:43.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:11:14.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:11:14.085 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:11:44.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:11:44.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:12:14.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:12:14.297 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:12:44.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:12:44.441 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:13:14.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:13:14.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:13:44.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:13:44.694 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:14:14.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:14:14.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:14:44.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:14:44.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:15:15.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:15:15.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:15:45.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:15:45.269 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:16:15.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:16:15.404 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:16:45.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:16:45.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:17:15.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:17:15.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:17:45.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:17:46.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:18:16.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:18:16.330 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:18:46.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:18:46.451 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:19:16.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:19:16.608 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:19:46.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:19:46.731 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:20:16.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:20:16.879 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:20:47.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:20:47.008 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:21:17.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:21:17.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:21:47.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:21:47.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:22:17.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:22:17.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:22:47.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:22:47.572 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:23:17.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:23:17.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:23:47.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:23:47.868 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:24:18.007 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:24:18.011 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:24:48.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:24:48.130 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:25:18.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:25:18.273 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:25:48.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:25:48.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:26:18.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:26:18.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:26:48.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:26:48.664 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:27:18.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:27:18.807 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:27:48.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:27:48.959 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:28:19.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:28:19.102 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:28:49.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:28:49.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:29:19.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:29:19.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:29:49.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:29:49.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:30:19.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:30:19.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:30:49.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:30:49.800 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:31:19.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:31:19.948 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:31:50.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:31:50.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:32:20.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:32:20.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:32:50.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:32:50.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:33:20.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:33:20.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:33:50.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:33:50.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:34:20.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:34:20.836 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:34:51.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:34:51.021 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:35:21.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:35:21.147 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:35:51.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:35:51.283 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:36:21.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:36:21.429 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:36:51.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:36:51.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:37:21.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:37:21.707 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:37:51.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:37:51.850 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:38:21.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:38:21.969 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:38:52.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:38:52.095 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:39:22.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:39:22.255 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:39:52.395 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:39:52.399 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:40:22.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:40:22.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:40:52.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:40:52.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:41:22.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:41:22.814 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:41:52.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:41:52.891 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:42:23.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:42:23.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:42:53.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:42:53.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:43:23.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:43:23.355 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:43:53.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:43:53.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:44:23.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:44:23.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:44:53.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:44:53.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:45:23.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:45:23.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:45:54.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:45:54.006 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:46:24.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:46:24.126 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:46:54.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:46:54.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:47:24.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:47:24.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:47:54.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:47:54.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:48:24.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:48:24.700 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:48:54.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:48:54.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:49:24.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:49:24.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:49:55.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:49:55.149 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:50:25.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:50:25.275 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:50:55.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:50:55.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:51:25.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:51:25.565 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:51:55.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:51:55.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:52:25.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:52:25.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:52:55.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:52:55.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:53:26.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:53:26.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:53:56.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:53:56.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:54:26.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:54:26.475 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:54:56.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:54:56.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:55:26.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:55:26.769 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:55:56.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:55:56.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:56:27.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:56:27.048 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:56:57.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:56:57.184 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:57:27.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:57:27.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:57:57.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:57:57.462 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:58:27.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:58:27.583 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:58:57.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:58:57.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:59:27.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:59:27.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:59:57.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T16:59:57.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:00:28.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:00:28.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:00:58.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:00:58.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:01:28.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:01:28.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:01:58.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:01:58.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:02:28.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:02:28.647 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:02:58.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:02:58.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:03:28.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:03:28.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:03:59.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:03:59.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:04:29.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:04:29.155 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:04:59.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:04:59.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:05:29.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:05:29.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:05:59.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:05:59.587 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:06:29.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:06:29.738 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:06:59.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:06:59.898 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:07:30.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:07:30.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:08:00.215 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:08:00.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:08:30.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:08:30.346 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:09:00.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:09:00.472 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:09:30.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:09:30.633 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:10:00.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:10:00.752 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:10:30.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:10:30.823 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:11:00.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:11:00.956 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:11:31.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:11:31.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:12:01.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:12:01.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:12:31.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:12:31.413 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:13:01.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:13:01.575 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:13:31.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:13:31.717 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:14:01.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:14:01.876 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:14:32.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:14:32.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:15:02.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:15:02.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:15:32.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:15:32.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:16:02.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:16:02.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:16:32.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:16:32.514 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-04T17:16:38.675 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-11-04T17:16:38.688 DEBUG:teuthology.task.console_log:Killing console logger for smithi003 2023-11-04T17:16:38.689 DEBUG:teuthology.task.console_log:Killing console logger for smithi188 2023-11-04T17:16:38.690 DEBUG:teuthology.exit:Finished running handlers